https://www2.econ.iastate.edu/tesfatsi/DeepLearningInNeuralNetworksOverview.JSchmidhuber2015.pdf

Deep learning in neural networks

Since emotions are expressed through a combination of verbal and non-verbal channels, a joint analysis of speech and gestures is required to understand expressive human communication. To facilitate such investigations, this paper describes a new corpus named the “interactive emotional dyadic motion capture database” (IEMOCAP), collected by the Speech Analysis and Interpretation Laboratory (SAIL) at the University of Southern California (USC). This database was recorded from ten actors in dyadic sessions with markers on the face, head, and hands, which provide detailed information about their facial expressions and hand movements during scripted and spontaneous spoken communication scenarios. The actors performed selected emotional scripts and also improvised hypothetical scenarios designed to elicit specific types of emotions (happiness, anger, sadness, frustration and neutral state). The corpus contains approximately 12 h of data. The detailed motion capture information, the interactive setting to elicit authentic emotions, and the size of the database make this corpus a valuable addition to the existing databases in the community for the study and modeling of multimodal and expressive human communication.

/pdf/iemocap-interactive-emotional-dyadic-motion-capture-database-2ffyt1sb5i.pdf

IEMOCAP: interactive emotional dyadic motion capture database

Work on voice sciences over recent decades has led to a proliferation of acoustic parameters that are used quite selectively and are not always extracted in a similar fashion. With many independent teams working in different research areas, shared standards become an essential safeguard to ensure compliance with state-of-the-art methods allowing appropriate comparison of results across studies and potential integration and combination of extraction and recognition systems. In this paper we propose a basic standard acoustic parameter set for various areas of automatic voice analysis, such as paralinguistic or clinical speech analysis. In contrast to a large brute-force parameter set, we present a minimalistic set of voice parameters here. These were selected based on a) their potential to index affective physiological changes in voice production, b) their proven value in former studies as well as their automatic extractability, and c) their theoretical significance. The set is intended to provide a common baseline for evaluation of future research and eliminate differences caused by varying parameter sets or even different implementations of the same parameters. Our implementation is publicly available with the openSMILE toolkit. Comparative evaluations of the proposed feature set and large baseline feature sets of INTERSPEECH challenges show a high performance of the proposed set in relation to its size.

/pdf/the-geneva-minimalistic-acoustic-parameter-set-gemaps-for-36spb4mksx.pdf

The Geneva Minimalistic Acoustic Parameter Set (GeMAPS) for Voice Research and Affective Computing

http://www.ffri.hr/~ibrdar/komunikacija/seminari/Ververidis,%202006%20-%20Emotional%20speech%20recognition.pdf

Emotional speech recognition: Resources, features, and methods

We present the Montreal Forced Aligner (MFA), a new opensource system for speech-text alignment. MFA is an update to the Prosodylab-Aligner, and maintains its key functionality of trainability on new data, as well as incorporating improved architecture (triphone acoustic models and speaker adaptation), and other features. MFA uses Kaldi instead of HTK, allowing MFA to be distributed as a stand-alone package, and to exploit parallel processing for computationally-intensive training and scaling to larger datasets. We evaluate MFA’s performance on aligning word and phone boundaries in English conversational and laboratory speech, relative to human-annotated boundaries, focusing on the effects of aligner architecture and training on the data to be aligned. MFA performs well relative to two existing open-source aligners with simpler architecture (Prosodylab-Aligner and FAVE), and both its improved architecture and training on data to be aligned generally result in more accurate boundaries.

Montreal Forced Aligner: Trainable Text-Speech Alignment Using Kaldi.

Multilingual processing of speech via web services

While the first open comparative challenges in the field of paralinguistics targeted more ‘conventional’ phenomena such as emotion, age, and gender, there still exists a multiplicity of not yet covered, but highly relevant speaker states and traits. The INTERSPEECH 2011 Speaker State Challenge thus addresses two new sub-challenges to overcome the usually low compatibility of results: In the Intoxication Sub-Challenge, alcoholisation of speakers has to be determined in two classes; in the Sleepiness Sub-Challenge, another two-class classification task has to be solved. This paper introduces the conditions, the Challenge corpora “Alcohol Language Corpus” and “Sleepy Language Corpus”, and a standard feature set that may be used. Further, baseline results are given. Index Terms: Speaker State Challenge, Intoxication, Sleepiness

/pdf/the-interspeech-2011-speaker-state-challenge-r7gishf9e5.pdf

The INTERSPEECH 2011 Speaker State Challenge

Automatic Segmentation" (MAUS) system labels and segments the phonetic constituents of spoken German in a manner similar to highly trained phoneticians. MAUS has been used to train automatic speech recognition (ASR) systems as well as to provide detailed statistical analyses of spontaneous speech (using the Verbmobil I and RVG I corpora). The MAUS system is a reliable, automatic means of testing linguistic hypotheses concerning the phonetic properties of spontaneous speech and should therefore play an important role in providing the sort of empirical data required to develop more realistic models of spoken language. 1. INTRODUCTION In many cases our scientific work with recorded non− prompted or even spontaneous German during the last 5 years ended in results that often differ from our text book knowledge of German phonetics. In the light of these observations it is my opinion that the speech sciences including phonetics should follow a new way (beside the traditional ways that are of course still to be pursued!) to comply with the problem that often the scientific models of speech differ significantly from reality. Therefore, in part 2 I will give some arguments for computational methods on the basis of large purpose− independent speech corpora. To give an example of this type of work the third section gives a brief description of the 'Munich Automatic Segmentation' (MAUS) method, while the last part will give three examples where results from MAUS were used in different experiments or applications. The first example is a statistical evaluation of well known assimilation processes at word boundaries; the second and third example describe experiments to improve Automatic Speech Recognition (ASR) by exploiting the knowledge about pronunciation from the MAUS segmentation.

Automatic Phonetic Transcription of Non-Prompted Speech

The CLARIN infrastructure aims at providing a technical infrastructure for language resources. In this context we present the design of a web service and a corresponding interface to provide easy access to an formerly only locally executable application for automatic segmentation and labeling called Munich AUtomatic Segmentation (MAUS). Using a standard description format we not only make the manual usage of those services possible, but also enable access from within other services or chaining engines like WebLicht. As an example for the integration of the web service in another application, we show the WebMAUS integration into ELAN.

Florian Schiel

Papers

Multilingual processing of speech via web services

The INTERSPEECH 2011 Speaker State Challenge

Automatic Phonetic Transcription of Non-Prompted Speech

Signal processing via web services: The use case WebMAUS

PROMISE - A Procedure for Multimodal Interactive System Evaluation