Emotion recognition in human-computer interaction

Tracing 20 years of progress in making machines hear our emotions based on speech signal properties.

Speech emotion recognition: two decades in a nutshell, benchmarks, and ongoing trends

Growth in the area of opinion mining and sentiment analysis has been rapid and aims to explore the opinions or text present on different platforms of social media through machine-learning techniques with sentiment, subjectivity analysis or polarity calculations. Despite the use of various machine-learning techniques and tools for sentiment analysis during elections, there is a dire need for a state-of-the-art approach. To deal with these challenges, the contribution of this paper includes the adoption of a hybrid approach that involves a sentiment analyzer that includes machine learning. Moreover, this paper also provides a comparison of techniques of sentiment analysis in the analysis of political views by applying supervised machine-learning algorithms such as Naive Bayes and support vector machines (SVM).

Machine Learning-Based Sentiment Analysis for Twitter Accounts

In this paper, global and local prosodic features extracted from sentence, word and syllables are proposed for speech emotion or affect recognition. In this work, duration, pitch, and energy values are used to represent the prosodic information, for recognizing the emotions from speech. Global prosodic features represent the gross statistics such as mean, minimum, maximum, standard deviation, and slope of the prosodic contours. Local prosodic features represent the temporal dynamics in the prosody. In this work, global and local prosodic features are analyzed separately and in combination at different levels for the recognition of emotions. In this study, we have also explored the words and syllables at different positions (initial, middle, and final) separately, to analyze their contribution towards the recognition of emotions. In this paper, all the studies are carried out using simulated Telugu emotion speech corpus (IITKGP-SESC). These results are compared with the results of internationally known Berlin emotion speech corpus (Emo-DB). Support vector machines are used to develop the emotion recognition models. The results indicate that, the recognition performance using local prosodic features is better compared to the performance of global prosodic features. Words in the final position of the sentences, syllables in the final position of the words exhibit more emotion discriminative information compared to the words and syllables present in the other positions.

Emotion recognition from speech using global and local prosodic features

Speech emotion recognition, a highly promising and exciting problem in the field of Human Computer Interaction, has been studied and analyzed over several decades. It concerns the task of recognizing a speaker’s emotions from their speech recordings. Recognizing emotions from speech can go a long way in determining a person’s physical and psychological state of well-being. In this work we performed emotion classification on three corpora — the Berlin EmoDB, the Indian Institute of Technology Kharagpur Simulated Emotion Hindi Speech Corpus (IITKGP-SEHSC), and the Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS). A combination of spectral features was extracted from them which was further processed and reduced to the required feature set. Ensemble learning has been proven to give superior performance compared to single estimators. We propose a bagged ensemble comprising of support vector machines with a Gaussian kernel as a viable algorithm for the problem at hand. We report the results obtained on the three datasets mentioned above.

Bagged support vector machines for emotion recognition from speech

In this paper, we propose a method for detecting the vowel onset points (VOPs) for low bit rate coded speech. VOP is the instant at which the onset of the vowel takes place in the speech signal. VOP plays an important role for the applications, such as consonant-vowel (CV) unit recognition and speech rate modification. The proposed VOP detection method is based on the spectral energy present in the glottal closure region of the speech signal. Speech coders considered to carry out this study are Global System for Mobile Communications (GSM) full rate, code-excited linear prediction (CELP), and mixed-excitation linear prediction (MELP). TIMIT database and CV units collected from the broadcast news corpus are used for evaluation. Performance of the proposed method is compared with existing methods, which uses the combination of evidence from the excitation source, spectral peaks energy, and modulation spectrum. The proposed VOP detection method has shown significant improvement in the performance compared to the existing method under clean as well as coded cases. The effectiveness of the proposed VOP detection method is analyzed in CV recognition by using VOP as an anchor point.

Vowel Onset Point Detection for Low Bit Rate Coded Speech

In this paper, simulated emotion Hindi speech corpus has been introduced for analyzing the emotions present in speech signals. The proposed database is recorded using professional artists from Gyanavani FM radio station, Varanasi, India. The speech corpus is collected by simulating eight different emotions using neutral (emotion free) text prompts. The emotions present in the database are anger, disgust, fear, happy, neutral, sad, sarcastic and surprise. This speech corpus is named as Indian Institute of Technology Kharagpur Simulated Emotion Hindi Speech Corpus (IITKGP-SEHSC). Emotion classification is performed on the proposed IITKGP-SEHSC using prosodic and spectral features. Mel frequency cepstral coefficients (MFCCs) are used to represent spectral information. Energy, pitch and duration are used to represent prosody information. The average emotion recognition performance using prosodic and spectral features are found to be around 77% and 81% for female speech utterances. This paper describes the design, acquisition, post processing and evaluation of the proposed speech corpus (IITKGP-SEHSC). The quality of the emotions expressed in the database is evaluated using subjective listening tests. The emotion recognition performance using subjective listening tests is observed to be around 74%. The results of subjective listening tests are grossly on par with the results obtained using prosodic analysis of the database.

IITKGP-SEHSC : Hindi Speech Corpus for Emotion Analysis

Background: Sarcasm detection in twitter is a very important task as it had helped in the analysis of tweets. With the help of sarcasm detection, companies could analyze the feelings of user about their products. This is helpful for companies, as the companies could improve their quality of product. Methods: For preprocessing of data TextBlob is used. TextBlob is a package installed in Natural Language Toolkit. The preprocessing steps include tokenization, part of speech tagging and parsing. The stop words are removed using python programming. The stop words corpus which consist of 2400 stop words and which is distributed with NLTK have been used. RapidMiner is used for finding polarity and subjectivity of tweets. TextBlob is used for finding the polarity and subjectivity confidence. Weka is used for calculating the accuracy of tweets based on Naive Bayes classifier and SVM classifiers. Findings: The paper provides the polarity of tweets which include whether the tweet is positive, negative or neutral. Polarity confidence and subjectivity confidence are also found. Accuracy of tweets are found using Naive Bayes and SVM classifiers. Applications: Sarcasm Detection could be helpful in analyzing the exact opinion of the user about a certain thing.

Proposed Approach for Sarcasm Detection in Twitter

In this paper we have analysed emotion recognition performance in speaker dependent, text dependent, text independent, speaker independent, language dependent and cross language emotion recognition from speech. These studies were carried out using Gaussian Mixture Model (GMM) and Hidden Markov Model (HMM) as classification models. IITKGP-SESC and IITKGP-SEHSC emotional speech corpora are used for carried out these studies. The emotions considered in this study are anger, disgust, fear, happy, neutral, sarcastic, and surprise. Mel Frequency Cepstral Coefficients (MFCCs) features are used for identifying the emotions. Emotion recognition performance of speaker dependent mode is better than speaker independent and cross language modes. From the results it is observed that emotion recognition performance depends on the speaker and language.

Jainath Yadav

Papers

Vowel Onset Point Detection for Low Bit Rate Coded Speech

IITKGP-SEHSC : Hindi Speech Corpus for Emotion Analysis

Proposed Approach for Sarcasm Detection in Twitter

Speaker dependent, speaker independent and cross language emotion recognition from speech using GMM and HMM

A survey of speech emotion recognition in natural environment