Vedanuj Goswami

TL;DR: This paper investigated the relationship between vision and language tasks by developing a large-scale, multi-task model, which culminates in a single model on 12 datasets from four broad categories of task including visual question answering, caption-based image retrieval, grounding referring expressions, and multimodal verification.

TL;DR: A conditional compute model based on Sparsely Gated Mixture of Experts that is trained on data obtained with novel and effective data mining techniques tailored for low-resource languages is developed, laying important groundwork towards realizing a universal translation system.

TL;DR: This work proposes a new challenge set for multimodal classification, focusing on detecting hate speech in multi-modal memes, constructed such that unimodal models struggle and only multimodAL models can succeed.

TL;DR: This work develops a large-scale, multi-task model that culminates in a single model on 12 datasets from four broad categories of task including visual question answering, caption-based image retrieval, grounding referring expressions, and multimodal verification and shows that finetuning task-specific models from this model can lead to further improvements, achieving performance at or above the state-of-the-art.

TL;DR: The authors proposed a new challenge set for multimodal classification, focusing on detecting hate speech in multi-modal memes, where difficult examples are added to the dataset to make it hard to rely on unimodal signals.