Vocabulary-Wide Credit Assignment for Training Image Captioning Models

TLDR: A Vocabulary-Critical Sequence Training (VCST) is proposed, which assigns every word in vocabulary an appropriate credit at each generation step and can be incorporated into existing RL methods for training image captioning models to achieve better results.

Abstract: Reinforcement learning (RL) algorithms have been shown to be efficient in training image captioning models. A critical step in RL algorithms is to assign credits to appropriate actions. There are mainly two classes of credit assignment methods in existing RL methods for image captioning, assigning a single credit for the whole sentence and assigning a credit to every word in the sentence. In this article, we propose a new credit assignment method which is orthogonal to the above two. It assigns every word in vocabulary an appropriate credit at each generation step. It is called vocabulary-wide credit assignment. Based on this we propose a Vocabulary-Critical Sequence Training (VCST). VCST can be incorporated into existing RL methods for training image captioning models to achieve better results. Extensive experiments with many popular models validated the effectiveness of VCST.