H
Hannah Rose Kirk
Researcher at University of Oxford
Publications - 5
Citations - 5
Hannah Rose Kirk is an academic researcher from University of Oxford. The author has contributed to research in topics: Test suite & Test (biology). The author has co-authored 5 publications.
Papers
More filters
Posted Content
Hatemoji: A Test Suite and Adversarially-Generated Dataset for Benchmarking and Detecting Emoji-based Hate.
TL;DR: HemojiCheck as discussed by the authors ) is a test suite of 3,930 short-form statements that allows the authors to evaluate how detection models perform on hateful language expressed with emoji.
Proceedings ArticleDOI
Memes in the Wild: Assessing the Generalizability of the Hateful Memes Challenge Dataset
Hannah Rose Kirk,Yennie Jun,Paulius Rauba,Gal Wachtel,Ruining Li,Xingjian Bai,Noah Broestl,Martin Doff-Sotta,Aleksandar Shtedritski,Yuki M. Asano +9 more
TL;DR: The authors collected hateful and non-hateful memes from Pinterest to evaluate out-of-sample performance on models pre-trained on the Facebook dataset and found that hateful memes are more diverse than traditional memes.
Posted Content
Bias Out-of-the-Box: An Empirical Analysis of Intersectional Occupational Biases in Popular Generative Language Models
Hannah Rose Kirk,Yennie Jun,Haider Iqbal,Elias Benussi,Filippo Volpin,Frédéric A. Dreyer,Aleksandar Shtedritski,Yuki M. Asano +7 more
TL;DR: The authors conducted an in-depth analysis of GPT-2, which is the most downloaded text generation model on HuggingFace, with over half a million downloads in the past month alone.
Posted Content
Memes in the Wild: Assessing the Generalizability of the Hateful Memes Challenge Dataset
Hannah Rose Kirk,Yennie Jun,Paulius Rauba,Gal Wachtel,Ruining Li,Xingjian Bai,Noah Broestl,Martin Doff-Sotta,Aleksandar Shtedritski,Yuki M. Asano +9 more
TL;DR: This article collected hateful and non-hateful memes from Pinterest to evaluate out-of-sample performance on models pre-trained on the Facebook dataset and found that memes in the wild differ in two key aspects: 1) Captions must be extracted via OCR, injecting noise and diminishing performance of multimodal models, and 2) Memes are more diverse than traditional memes, including screenshots of conversations or text on a plain background.
Posted Content
How True is GPT-2? An Empirical Analysis of Intersectional Occupational Biases.
Hannah Rose Kirk,Yennie Jun,Haider Iqbal,Elias Benussi,Filippo Volpin,Frédéric A. Dreyer,Aleksandar Shtedritski,Yuki M. Asano +7 more
TL;DR: This paper analyzed the occupational biases of a popular generative language model, GPT-2, intersecting gender with five protected categories: religion, sexuality, ethnicity, political affiliation, and name origin.