Open AccessPosted Content
The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memes
Douwe Kiela,Hamed Firooz,Aravind Mohan,Vedanuj Goswami,Amanpreet Singh,Pratik Ringshia,Davide Testuggine +6 more
Reads0
Chats0
TLDR
The authors proposed a new challenge set for multimodal classification, focusing on detecting hate speech in multi-modal memes, where difficult examples are added to the dataset to make it hard to rely on unimodal signals.Abstract:
This work proposes a new challenge set for multimodal classification, focusing on detecting hate speech in multimodal memes. It is constructed such that unimodal models struggle and only multimodal models can succeed: difficult examples ("benign confounders") are added to the dataset to make it hard to rely on unimodal signals. The task requires subtle reasoning, yet is straightforward to evaluate as a binary classification problem. We provide baseline performance numbers for unimodal models, as well as for multimodal models with various degrees of sophistication. We find that state-of-the-art methods perform poorly compared to humans (64.73% vs. 84.7% accuracy), illustrating the difficulty of the task and highlighting the challenge that this important problem poses to the community.read more
Citations
More filters
Posted Content
Multimodal Learning for Hateful Memes Detection
Yi Zhou,Zhenhao Chen +1 more
TL;DR: In this article, the authors focus on multimodal hateful memes detection and propose a novel method that incorporates the image captioning process into the memes detection process, which achieves promising results on the Hateful Memes Detection Challenge.
Findings of the Shared Task on Troll Meme Classification in Tamil
TL;DR: In this article, the authors provided a resource (TamilMemes) that could be used to train a system capable of identifying a troll meme in the Tamil language and the system with the weighted average F1-score of 0.55 secured the first rank.
Proceedings ArticleDOI
Disentangling Hate in Online Memes
TL;DR: DisMultiHate as mentioned in this paper disentangles target entities in multimodal memes to improve hateful content classification and explainability, achieving state-of-the-art performance in hateful meme classification task.
Journal ArticleDOI
Multimodal Hate Speech Detection in Greek Social Media
TL;DR: This study presents a new multimodal approach to hate speech detection by combining Computer Vision and Natural Language processing models for abusive context detection on Twitter messages, focusing on hateful, xenophobic, and racist speech in Greek aimed at refugees and migrants.
Posted Content
A Multimodal Memes Classification: A Survey and Open Research Issues
TL;DR: This study presents a clear road-map for the Machine Learning (ML) research community to implement and enhance memes classification techniques, and proposes a generalized framework for VL problems.
References
More filters
Proceedings ArticleDOI
Deep Residual Learning for Image Recognition
TL;DR: In this article, the authors proposed a residual learning framework to ease the training of networks that are substantially deeper than those used previously, which won the 1st place on the ILSVRC 2015 classification task.
Proceedings Article
Adam: A Method for Stochastic Optimization
Diederik P. Kingma,Jimmy Ba +1 more
TL;DR: This work introduces Adam, an algorithm for first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments, and provides a regret bound on the convergence rate that is comparable to the best known results under the online convex optimization framework.
Book ChapterDOI
Microsoft COCO: Common Objects in Context
Tsung-Yi Lin,Michael Maire,Serge Belongie,James Hays,Pietro Perona,Deva Ramanan,Piotr Dollár,C. Lawrence Zitnick +7 more
TL;DR: A new dataset with the goal of advancing the state-of-the-art in object recognition by placing the question of object recognition in the context of the broader question of scene understanding by gathering images of complex everyday scenes containing common objects in their natural context.
Proceedings ArticleDOI
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
TL;DR: BERT as mentioned in this paper pre-trains deep bidirectional representations from unlabeled text by jointly conditioning on both left and right context in all layers, which can be fine-tuned with just one additional output layer to create state-of-the-art models for a wide range of tasks.
Posted Content
Adam: A Method for Stochastic Optimization
Diederik P. Kingma,Jimmy Ba +1 more
TL;DR: In this article, the adaptive estimates of lower-order moments are used for first-order gradient-based optimization of stochastic objective functions, based on adaptive estimate of lowerorder moments.