The measurement of meaning

The cultural origins of human cognition

A survey of context modelling and reasoning techniques

The RAVDESS is a validated multimodal database of emotional speech and song. The database is gender balanced consisting of 24 professional actors, vocalizing lexically-matched statements in a neutral North American accent. Speech includes calm, happy, sad, angry, fearful, surprise, and disgust expressions, and song contains calm, happy, sad, angry, and fearful emotions. Each expression is produced at two levels of emotional intensity, with an additional neutral expression. All conditions are available in face-and-voice, face-only, and voice-only formats. The set of 7356 recordings were each rated 10 times on emotional validity, intensity, and genuineness. Ratings were provided by 247 individuals who were characteristic of untrained research participants from North America. A further set of 72 participants provided test-retest data. High levels of emotional validity and test-retest intrarater reliability were reported. Corrected accuracy and composite "goodness" measures are presented to assist researchers in the selection of stimuli. All recordings are made freely available under a Creative Commons license and can be downloaded at https://doi.org/10.5281/zenodo.1188976.

/pdf/the-ryerson-audio-visual-database-of-emotional-speech-and-jx71tsx3ts.pdf

The Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS): A dynamic, multimodal set of facial and vocal expressions in North American English

Emotion and Meaning in Music

When speaking or singing, vocalists often move their heads in an expressive fashion, yet the influence of emotion on vocalists' head motion is unknown. Using a comparative speech/song task, we examined whether vocalists' intended emotions influence head movements and whether those movements influence the perceived emotion. In Experiment 1, vocalists were recorded with motion capture while speaking and singing each statement with different emotional intentions (very happy, happy, neutral, sad, very sad). Functional data analyses showed that head movements differed in translational and rotational displacement across emotional intentions, yet were similar across speech and song, transcending differences in F0 (varied freely in speech, fixed in song) and lexical variability. Head motion specific to emotional state occurred before and after vocalizations, as well as during sound production, confirming that some aspects of movement were not simply a by-product of sound production. In Experiment 2, observers accurately identified vocalists' intended emotion on the basis of silent, face-occluded videos of head movements during speech and song. These results provide the first evidence that head movements encode a vocalist's emotional intent and that observers decode emotional information from these movements. We discuss implications for models of head motion during vocalizations and applied outcomes in social robotics and automated emotion recognition.

Head movements encode emotions during speech and song.

Two experiments examined the effects of repetition on listeners’ emotional response to music. Listeners heard recordings of orchestral music that contained a large section repeated twice. The music had a symmetric phrase structure (same-length phrases) in Experiment 1 and an asymmetric phrase structure (different-length phrases) in Experiment 2, hypothesized to alter the predictability of sensitivity to musical repetition. Continuous measures of arousal and valence were compared across music that contained identical repetition, variation (related), or contrasting (unrelated) structure. Listeners’ emotional arousal ratings differed most for contrasting music, moderately for variations, and least for repeating musical segments. A computational model for the detection of repeated musical segments was applied to the listeners’ emotional responses. The model detected the locations of phrase boundaries from the emotional responses better than from performed tempo or physical intensity in both experiments. These findings indicate the importance of repetition in listeners’ emotional response to music and in the perceptual segmentation of musical structure.

/pdf/emotional-response-to-musical-repetition-vrptxlu0bc.pdf

Emotional response to musical repetition.

Vocal training through singing and acting lessons is known to modify acoustic parameters of the voice While the effects of singing training have been well documented, the role of acting experience on the singing voice remains unclear In two experiments, we used linear mixed models to examine the relationships between the relative amounts of acting and singing experience on the acoustics and perception of the male singing voice In Experiment 1, twelve male vocalists were recorded while singing with five different emotions, each with two intensities Acoustic measures of pitch accuracy, jitter, and harmonics-to-noise (HNR) ratio were examined Decreased pitch accuracy and increased jitter, indicative of a lower ‘voice quality’, were associated with more years of acting experience, while increased pitch accuracy was associated with more years of singing lessons We hypothesized that the acoustic deviations exhibited by more experienced actors was an intentional technique to increase the genuineness or truthfulness of their emotional expressions In Experiment 2, listeners rated vocalists’ emotional genuineness Vocalists with more years of acting experience were rated as more genuine than vocalists with less acting experience No relationship was reported for singing training Increased genuineness was associated with decreased pitch accuracy, increased jitter, and a higher harmonics-to-noise ratio These effects may represent a shifting of priorities by male vocalists with acting experience to emphasize emotional genuineness over pitch accuracy or voice quality in their singing performances

/pdf/the-influence-of-vocal-training-and-acting-experience-on-3casrt7p6m.pdf

The influence of vocal training and acting experience on measures of voice quality and emotional genuineness.

Speech and song are universal forms of vocal expression that reflect distinct channels of communication. While these two forms of expression share a common means of sound production, differences in the acoustic properties of speech and song have not received significant attention. Here, we present evidence of acoustic differences in the speaking and singing voice. Twenty-four actors were recorded while speaking and singing different statements with five emotions, two emotional intensities, and two repetitions. Acoustic differences of speech and song were reported in several acoustic parameters, including vocal loudness, spectral properties, and vocal quality. Interestingly, emotion was conveyed similarly in many acoustic features across speech and song. These results demonstrate the entwined nature of speech and song, and provide evidence in support of the shared emergence speech and song as a form of early proto-language. These recordings form part of our new Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS) that will be freely released in 2013.

Acoustic differences in the speaking and singing voice

The rigors of establishing innateness and domain specificity pose challenges to adaptationist models of music evolution. In articulating a series of constraints, the authors of the target articles provide strategies for investigating the potential origins of music. We propose additional approaches for exploring theories based on exaptation. We discuss a view of music as a multimodal system of engaging with affect, enabled by capacities of symbolism and a theory of mind.

/pdf/multimodal-affective-interaction-a-comment-on-musical-2qilkge8e6.pdf

Steven R. Livingstone

Papers

Head movements encode emotions during speech and song.

Emotional response to musical repetition.

The influence of vocal training and acting experience on measures of voice quality and emotional genuineness.

Acoustic differences in the speaking and singing voice

Multimodal affective interaction: a comment on musical origins