A multimodal data set for the analysis of human affective states was presented and a novel method for stimuli selection is proposed using retrieval by affective tags from the last.fm website, video highlight detection, and an online assessment tool.
Abstract:
We present a multimodal data set for the analysis of human affective states. The electroencephalogram (EEG) and peripheral physiological signals of 32 participants were recorded as each watched 40 one-minute long excerpts of music videos. Participants rated each video in terms of the levels of arousal, valence, like/dislike, dominance, and familiarity. For 22 of the 32 participants, frontal face video was also recorded. A novel method for stimuli selection is proposed using retrieval by affective tags from the last.fm website, video highlight detection, and an online assessment tool. An extensive analysis of the participants' ratings during the experiment is presented. Correlates between the EEG signal frequencies and the participants' ratings are investigated. Methods and results are presented for single-trial classification of arousal, valence, and like/dislike ratings using the modalities of EEG, peripheral physiological signals, and multimedia content analysis. Finally, decision fusion of the classification results from different modalities is performed. The data set is made publicly available and we encourage other researchers to use it for testing their own affective state estimation methods.
TL;DR: An emotion detection method for music videos using central and peripheral nervous system physiological signals as well as multimedia content analysis and the performance of the personalized emotion detection is shown to be significantly superior to a random regressor.
TL;DR: This paper presents on-going work on using Deep Belief Network (DBN) to automatically extract high-level features from raw EEG signals and shows that the learned features perform comparably to the use of manually generated features for emotion recognition.
TL;DR: A new facial expression database containing spontaneous expressions of both male and female participants of Indian origin is proposed and established, which will help in the development and validation of algorithms for recognition of spontaneous expressions.
TL;DR: This work aims at creating an ontology-based context model for emotion recognition using EEG, which completely implements the loop body of the W2T data cycle once: from low-level EEG feature acquisition to emotion recognition.
TL;DR: A detailed survey of the current literature and outline the scientific work conducted on brain biometric systems is provided, including an up-to-date review of state-of-the-art acquisition, collection, processing, and analysis of brainwave signals, publicly available databases, feature extraction and selection, and classifiers.
TL;DR: Reports of affective experience obtained using SAM are compared to the Semantic Differential scale devised by Mehrabian and Russell (An approach to environmental psychology, 1974), which requires 18 different ratings.
TL;DR: Key issues in affective computing, " computing that relates to, arises from, or influences emotions", are presented and new applications are presented for computer-assisted learning, perceptual information retrieval, arts and entertainment, and human health and interaction.
Q1. What are some common features used to characterize affect in music?
tempo, Mel-frequency cepstral coefficients (MFCC), pitch, zero crossing rate are amongst common features which have been used to characterize affect in music.
Q2. What are the contributions mentioned in the paper "Deap: a database for emotion analysis using physiological signals" ?
The authors present a multimodal data set for the analysis of human affective states. An extensive analysis of the participants ' ratings during the experiment is presented.
Q3. What test was used to test for significance?
To test for significance, an independent one-sample t-test was performed, comparing the F1-distribution over participants to the 0.5 baseline.
Q4. What other features have been shown to be correlated with valence?
There are other content features such as color variance and key lighting that have been shown to be correlated with valence [30].
Q5. What are the four quadrants of the valence-arousal space?
The valence-arousal space can be subdivided into 4 quadrants, namely low arousal/low valence (LALV), low arousal/high valence (LAHV), high arousal/low valence (HALV) and high arousal/high valence (HAHV).
Q6. How many videos were selected via Last.fm affective tags?
Of the 40 selected videos, 17 were selected via Last.fm affective tags, indicating that useful stimuli can be selected via this method.
Q7. What are the common types of emotional information used for emotion assessment?
Physiological signals are also known to include emotional information that can be used for emotion assessment but they have received less attention.
Q8. What are the two widely available databases for emotion assessment?
To the best of their knowledge, the only publicly available multi-modal emotional databases which includes both physiological responses and facial expressions are the enterface 2005 emotional database and MAHNOB HCI [4], [5].
Q9. What was the emotional highlight score of the i-th segment ei?
The emotional highlight score of the i-th segment ei was computed using the following equation:ei = √ a2 i + v2 i (1)The arousal, ai, and valence, vi, were centered.
Q10. How did the participants rate their familiarity with the songs?
after the experiment, participants were asked to rate their familiarity with each of the songs on a scale of 1 (”Never heard it before the experiment”) to 5 (”Knew the song very well”).
Q11. What was the arousal and valence level of each video?
The participants rated arousal and valence levels and the EEG and physiological signals for each video were classified into low/high arousal/valence classes.
The paper mentions several emotion datasets, including one recorded by Healey with physiological signals and others with speech, visual, or audiovisual data.