Speech emotion recognition is a vital and challenging task that the feature extraction plays a significant role in the SER performance. With the development of deep learning, we put our eyes on the structure of end-to-end and authenticate the algorithm that is extraordinary effective. In this paper, we introduce a novel architecture ADRNN (dilated CNN with residual block and BiLSTM based on the attention mechanism) to apply for the speech emotion recognition which can take advantage of the strengths of diverse networks and overcome the shortcomings of utilizing alone, and are evaluated in the popular IEMOCAP database and Berlin EMODB corpus. Dilated CNN can assist the model to acquire more receptive fields than using the pooling layer. Then, the skip connection can keep more historic info from the shallow layer and BiLSTM layer are adopted to learn long-term dependencies from the learned local features. And we utilize the attention mechanism to enhance further extraction of speech features. Furthermore, we improve the loss function to apply softmax together with the center loss that achieves better classification performance. As emotional dialogues are transformed of the spectrograms, we pick up the values of the 3-D Log-Mel spectrums from raw signals and put them into our proposed algorithm and obtain a notable performance to get the 74.96% unweighted accuracy in the speaker-dependent and the 69.32% unweighted accuracy in the speaker-independent experiment. It is better than the 64.74% from previous state-of-the-art methods in the spontaneous emotional speech of the IEMOCAP database. In addition, we propose the networks that achieve recognition accuracies of 90.78% and 85.39% on Berlin EMODB of speaker-dependent and speaker-independent experiment respectively, which are better than the accuracy of 88.30% and 82.82% obtained by previous work. For validating the robustness and generalization, we also make an experiment for cross-corpus between above databases and get the preferable 63.84% recognition accuracy in final.

/pdf/speech-emotion-recognition-from-3d-log-mel-spectrograms-with-fpm0b8dg8e.pdf

Speech Emotion Recognition From 3D Log-Mel Spectrograms With Deep Learning Network

Much work is focused upon music genre recognition (MGR) from audio recordings, symbolic data, and other modalities. While reviews have been written of some of this work before, no survey has been made of the approaches to evaluating approaches to MGR. This paper compiles a bibliography of work in MGR, and analyzes three aspects of evaluation: experimental designs, datasets, and figures of merit.

/pdf/a-survey-of-evaluation-in-music-genre-recognition-29v3l1icya.pdf

A survey of evaluation in music genre recognition

We argue that an evaluation of system behavior at the level of the music is required to usefully address the fundamental problems of music genre recognition (MGR), and indeed other tasks of music information retrieval, such as autotagging. A recent review of works in MGR since 1995 shows that most (82 %) measure the capacity of a system to recognize genre by its classification accuracy. After reviewing evaluation in MGR, we show that neither classification accuracy, nor recall and precision, nor confusion tables, necessarily reflect the capacity of a system to recognize genre in musical signals. Hence, such figures of merit cannot be used to reliably rank, promote or discount the genre recognition performance of MGR systems if genre recognition (rather than identification by irrelevant confounding factors) is the objective. This motivates the development of a richer experimental toolbox for evaluating any system designed to intelligently extract information from music signals.

/pdf/classification-accuracy-is-not-enough-5c5sdluzgy.pdf

Classification accuracy is not enough

Hidden Markov models (HMMs) have a long tradition in automatic speech recognition (ASR) due to their capability of capturing temporal dynamic characteristics of speech. For emotion recognition from speech, three HMM based architectures are investigated and compared throughout the current paper, namely, the Gaussian mixture model based HMMs (GMM-HMMs), the subspace based Gaussian mixture model based HMMs (SGMM-HMMs) and the hybrid deep neural network HMMs (DNN-HMMs). Extensive emotion recognition experiments are carried out on these three architectures on the CASIA corpus, the Emo-DB corpus and the IEMOCAP database, respectively, and results are compared with those of state-of-the-art approaches. These HMM based architectures prove capable of constituting an effective model for speech emotion recogntion. Also, the modeling accuracy is further enhanced by incorporating various advanced techniques from the ASR area. In particular, among all of the architectures, the SGMM-HMMs achieve the best performance in most of the experiments.

Revisiting Hidden Markov Models for Speech Emotion Recognition

Discrete speech emotion recognition (SER), the assignment of a single emotion label to an entire speech utterance, is typically performed as a sequence-to-label task. This approach, however, is limited, in that it can result in models that do not capture temporal changes in the speech signal, including those indicative of a particular emotion. One potential solution to overcome this limitation is to model SER as a sequence-to-sequence task instead. In this regard, we have developed an attention-based bidirectional long short-term memory (BLSTM) neural network in combination with a connectionist temporal classification (CTC) objective function (Attention-BLSTM-CTC) for SER. We also assessed the benefits of incorporating two contemporary attention mechanisms, namely component attention and quantum attention, into the CTC framework. To the best of the authors’ knowledge, this is the first time that such a hybrid architecture has been employed for SER. We demonstrated the effectiveness of our approach on the Interactive Emotional Dyadic Motion Capture (IEMOCAP) and FAU-Aibo Emotion corpora. The experimental results demonstrate that our proposed model outperforms current state-of-the-art approaches.

/pdf/attention-enhanced-connectionist-temporal-classification-for-1qgr5tj3pk.pdf

Attention-Enhanced Connectionist Temporal Classification for Discrete Speech Emotion Recognition.

To model the categorical speech emotion recognition task in a temporal manner, the first challenge arising is how to transfer the categorical label for each utterance into a label sequence. To settle this, we make a hypothesis that an utterance is consisting of emotional and non-emotional segments, and these non-emotional segments correspond to silent regions, short pauses, transitions between phonemes, unvoiced phonemes, etc. With this hypothesis, we propose to treat an utterance’s label sequence as a chain of two states: the emotional state denoting the emotional frame and Null denoting the non-emotional frame. Then, we exploit a recurrent neural network based connectionist temporal classification model to automatically label and align an utterance’s emotional segments with emotional labels, while non-emotional segments with Nulls. Experimental results on the IEMOCAP corpus validate our hypothesis and also demonstrate the effectiveness of our proposed method compared to the state-of-the-art algorithms.

Towards Temporal Modelling of Categorical Speech Emotion Recognition.

1 Educational Physiology Laboratory, The University of Tokyo, Tokyo, Japan, Department of Neurology, Children’s Hospital of Chongqing Medical University, Chongqing, China, 3 School of Computer Science and Technology, Harbin Institute of Technology, Harbin, China, 4 Institute of Information Photonics and Optical Communications, Beijing University of Posts and Telecommunications, Beijing, China, 5 School of Computer Science and Technology, Fudan University, Shanghai, China, 6 Shanghai Computer Music Association (SCMA), Shanghai, China, 7 School of Communication and Information Engineering, Shanghai University, Shanghai, China, 8 School of Mechanical and Electrical Engineering, Beijing University of Chemical Technology, Beijing, China, 9 School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu, China, Department of Computer Science and Technology, Anhui University of Finance and Economics, Bengbu, China, Department of Information and Computer, Taiyuan University of Technology, Taiyuan, China, 12 Ennova Health, Langfang, China, Department of Music Technology, Shenyang Conservatory of Music, Shenyang, China, 14 Speech Group, AI Lab, AI Department, Xiaomi, Beijing, China, 15 School of Precision Instrument and Opto-Electronics Engineering, Tianjin University, Tianjin, China, GLAM – Group on Language, Audio & Music, Imperial College London, London, United Kingdom, Chair of Embedded Intelligence for Health Care and Wellbeing, Augsburg University, Augsburg, Germany, 18 audEERING GmbH, Gilching, Germany

/pdf/computer-audition-for-healthcare-opportunities-and-m1d81jhryk.pdf

Computer Audition for Healthcare: Opportunities and Challenges

State-of-the-art dimensional speech emotion recognition systems are trained using continuously labelled instances. The data labelling process is labour intensive and time-consuming. In this paper, we propose to apply active learning to reduce according efforts: The unlabelled instances are evaluated automatically, and only the most informative ones are intelligently picked by an informativeness measure function for a human to label. Specifically, we estimate the informativeness of each unlabelled instance based on a binary-classification confidence score for an emotion being predicted to be negative or positive on a given emotional dimension. For verification, we consider a pool-based and a stream-based scenario run on part of the continuous AVEC 2012 task to demonstrate the feasibility of the proposed approach in practice. In the result, our approach requires significantly less human labelled data instances to reach a given performance than passive learning does in both scenarios.

Active Learning for Dimensional Speech Emotion Recognition

To investigate the effect of tunnel slope on hot gas movement and smoke distribution in a slopping tunnel fire, a series of tunnel fire models are built by fire dynamics simulator (FDS), with a slope varies from 0 to 10%. Parameters such as ceiling temperature and airflow velocity are measured. The results indicate that the relationship between smoke back-layering length and tunnel slope can be described as an exponential function. The smoke temperature at the downstream exit first increased and then decreased with a higher slope. The airflow velocity at downstream outlet increased nonlinearity when tunnel slope was less than 8%. In the slope tunnel, the fire smoke spread process can be divided into three stages. Fire smoke spreads upstream to the peak distance, subsequently, the upstream smoke layer decreases gradually, the tunnel fire reaches a quasi-steady state. The backflow characteristics of smoke in sloped tunnels are coupled with the downstream length and outlet smoke temperature. In the initial stage of a slope tunnel fire, smoke spreads upstream for a long distance, endangering human health. 

A Numerical Study on Smoke Behaviors in Inclined Tunnel Fires under Natural Ventilation

As one of the most well-known music elements, music key, which reveals an important feature in music transcription, structure analysis and mood comprehension, is always an essential theoretical construct of music As a result, key finding is becoming a popular topic of Music Information Retrieval (MIR) In this paper, we propose a novel approach with good robustness to detect keys in polyphonic music from a view of pitch class distribution theory A signal transformation with musical representation-Constant Q transform (CQT) is firstly applied to music audio for spectrum analysis Then onset detection and pitch tuning are introduced in order to ensure robustness Finally, a weighted harmonic structure pattern-pitch class distribution matrix (PCDM) is extracted as feature for key classification PCDM contains both pitch class information and chord structure, and it is based on pitch class distribution view Considering the classifier, a neural network is applied to model the pitch class distribution and complete the task of key recognition Also, a key smoothing method makes proposed method capable of processing modulation and reducing key fluctuation Experiments showed that the proposed strategy can reach a good performance in polyphonic music at a relatively lower computational cost, and proved our strategy to be quite promising

Haifeng Li

Papers

Towards Temporal Modelling of Categorical Speech Emotion Recognition.

Computer Audition for Healthcare: Opportunities and Challenges

Active Learning for Dimensional Speech Emotion Recognition

A Numerical Study on Smoke Behaviors in Inclined Tunnel Fires under Natural Ventilation

A music key detection method based on pitch class distribution theory