WASSA-2017 Shared Task on Emotion Intensity

TLDR: The first shared task on detecting the intensity of emotion felt by the speaker of a tweet and the first datasets of tweets annotated for anger, fear, joy, and sadness intensities using a technique called best–worst scaling (BWS) are presented.

Abstract: We present the first shared task on de- tecting the intensity of emotion felt by the speaker of a tweet. We create the first datasets of tweets annotated for anger, fear, joy, and sadness intensities using a technique called best–worst scal- ing (BWS). We show that the annota- tions lead to reliable fine-grained intensity scores (rankings of tweets by intensity). The data was partitioned into training, de- velopment, and test sets for the compe- tition. Twenty-two teams participated in the shared task, with the best system ob- taining a Pearson correlation of 0.747 with the gold intensity scores. We summarize the machine learning setups, resources, and tools used by the participating teams, with a focus on the techniques and re- sources that are particularly useful for the task. The emotion intensity dataset and the shared task are helping improve our under- standing of how we convey more or less intense emotions through language.