Survey on speech emotion recognition: Features, classification schemes, and databases

http://www.cs.columbia.edu/~julia/papers/Scherer03.pdf

Vocal communication of emotion: a review of research paradigms

Work on voice sciences over recent decades has led to a proliferation of acoustic parameters that are used quite selectively and are not always extracted in a similar fashion. With many independent teams working in different research areas, shared standards become an essential safeguard to ensure compliance with state-of-the-art methods allowing appropriate comparison of results across studies and potential integration and combination of extraction and recognition systems. In this paper we propose a basic standard acoustic parameter set for various areas of automatic voice analysis, such as paralinguistic or clinical speech analysis. In contrast to a large brute-force parameter set, we present a minimalistic set of voice parameters here. These were selected based on a) their potential to index affective physiological changes in voice production, b) their proven value in former studies as well as their automatic extractability, and c) their theoretical significance. The set is intended to provide a common baseline for evaluation of future research and eliminate differences caused by varying parameter sets or even different implementations of the same parameters. Our implementation is publicly available with the openSMILE toolkit. Comparative evaluations of the proposed feature set and large baseline feature sets of INTERSPEECH challenges show a high performance of the proposed set in relation to its size.

/pdf/the-geneva-minimalistic-acoustic-parameter-set-gemaps-for-36spb4mksx.pdf

The Geneva Minimalistic Acoustic Parameter Set (GeMAPS) for Voice Research and Affective Computing

The RAVDESS is a validated multimodal database of emotional speech and song. The database is gender balanced consisting of 24 professional actors, vocalizing lexically-matched statements in a neutral North American accent. Speech includes calm, happy, sad, angry, fearful, surprise, and disgust expressions, and song contains calm, happy, sad, angry, and fearful emotions. Each expression is produced at two levels of emotional intensity, with an additional neutral expression. All conditions are available in face-and-voice, face-only, and voice-only formats. The set of 7356 recordings were each rated 10 times on emotional validity, intensity, and genuineness. Ratings were provided by 247 individuals who were characteristic of untrained research participants from North America. A further set of 72 participants provided test-retest data. High levels of emotional validity and test-retest intrarater reliability were reported. Corrected accuracy and composite "goodness" measures are presented to assist researchers in the selection of stimuli. All recordings are made freely available under a Creative Commons license and can be downloaded at https://doi.org/10.5281/zenodo.1188976.

/pdf/the-ryerson-audio-visual-database-of-emotional-speech-and-jx71tsx3ts.pdf

The Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS): A dynamic, multimodal set of facial and vocal expressions in North American English

https://dspace.stir.ac.uk/bitstream/1893/25490/1/affective-computing-review.pdf

A review of affective computing

The article describes a database of emotional speech. Ten actors (5 female and 5 male) simulated the emotions, producing 10 German utterances (5 short and 5 longer sentences) which could be used in everyday communication and are interpretable in all applied emotions. The recordings were taken in an anechoic chamber with high-quality recording equipment. In addition to the sound electro-glottograms were recorded. The speech material comprises about 800 sentences (seven emotions * ten actors * ten sentences + some second versions). The complete database was evaluated in a perception test regarding the recognisability of emotions and their naturalness. Utterances recognised better than 80% and judged as natural by more than 60% of the listeners were phonetically labelled in a narrow transcription with special markers for voice-quality, phonatory and articulatory settings and articulatory features. The database can be accessed by the public via the internet (http://www.expressive-speech.net/emodb/).

/pdf/a-database-of-german-emotional-speech-2utsa757sb.pdf

A database of German emotional speech.

Emotions influence a person’s way of speaking, and it is possible to identify the emotional state of a speaker by merely listening to spoken utterances. The purpose of this study is to distinguish between basic emotions by prosodic features, in particular by characteristics of the fundamental frequency (F0). In addition to the measurement of global features (e.g. average F0 or frequency range) a number of more detailed features are examined. These parameters among others are the direction and steepness of F0 changes on syllable level and the F0 contour over whole utterances after a stylization procedure. Furthermore, the intrasyllabic F0 contour without stylization is investigated. The majority of the results can be used to distinguish between the emotions fear, anger, happiness, sadness, boredom, and disgust.

/pdf/f0-contours-in-emotional-speech-z954lonerd.pdf

F0-contours in emotional speech

In this study – which is part of an extensive investigation of the prosodic features of emotional speech – global trends of fundamental frequency were examined. The primary goal was to test the use of this parameter to characterize a specific set of emotions (happiness, anger, anxiety, sadness, disgust and boredom). Global F0 trend was measured in the form of the gradient of the linear regression in order to avoid the disadvantages associated with the determination of base and top lines commonly used to examine declination. A secondary goal was to bring a new argument into the discussion of causes for declination and its degree of influence on speech production. Results show a significant steeper downward trend for boredom than for neutral speech and a significant smaller falling global trend for all other emotions than for neutral speech. While the global trend seems to be appropriate especially for the description of boredom and sadness and to a slightly lesser extent also for anxiety and disgust, it is almost meaningless for the emotions happiness and anger. In happy and angry utterances the pitch accents are strong enough to hide the global downward trend completely. The results for each emotion are discussed in detail with respect to specific physiological constraints, as well as auditory impression and other prosodic features typical for these emotions.

Global Trend of Fundamental Frequency in Emotional Speech

The present study aims at examining vocal expression of emotion. Emotionally loaded speech material produced by actors was analyzed with reference to the accuracy of articulation as well as to the duration of syllables and segments. It was investigated whether the vocal expression of several emotions (anger, happiness, fear and sadness) differ from one another and from neutral speech with respect to the durational and qualitative reduction of articulation. The obtained results suggest that accuracy of articulation and durational aspects are useful parameters to characterize especially the emotions anger, fear and sadness.

Astrid Paeschke

Papers

A database of German emotional speech.

F0-contours in emotional speech

Global Trend of Fundamental Frequency in Emotional Speech

Articulatory reduction in emotional speech.