Trends in audio signal feature extraction methods

This paper proposes a system that allows recognizing a person's emotional state starting from audio signal registrations. The provided solution is aimed at improving the interaction among humans and computers, thus allowing effective human-computer intelligent interaction. The system is able to recognize six emotions(anger, boredom, disgust, fear, happiness, and sadness) and the neutral state. This set of emotional states is widely used for emotion recognition purposes. It also distinguishes a single emotion versus all the other possible ones, as proven in the proposed numerical results. The system is composed of two subsystems: 1) gender recognition(GR) and 2) emotion recognition(ER). The experimental analysis shows the performance in terms of accuracy of the proposed ER system. The results highlight that the a priori knowledge of the speaker's gender allows a performance increase. The obtained results show also that the features selection adoption assures a satisfying recognition rate and allows reducing the employed features. Future developments of the proposed solution may include the implementation of this system over mobile devices such as smartphones.

Gender-Driven Emotion Recognition Through Speech Signals For Ambient Intelligence Applications

In this work, source, system, and prosodic features of speech are explored for characterizing and classifying the underlying emotions. Different speech features contribute in different ways to express the emotions, due to their complementary nature. Linear prediction residual samples chosen around glottal closure regions, and glottal pulse parameters are used to represent excitation source information. Linear prediction cepstral coefficients extracted through simple block processing and pitch synchronous analysis represent the vocal tract information. Global and local prosodic features extracted from gross statistics and temporal dynamics of the sequence of duration, pitch, and energy values represent the prosodic information. Emotion recognition models are developed using above mentioned features separately, and in combination. Simulated Telugu emotion database (IITKGP-SESC) is used to evaluate the proposed features. The emotion recognition results obtained using IITKGP-SESC are compared with the results of internationally known Berlin emotion speech database (Emo-DB). Autoassociative neural networks, Gaussian mixture models, and support vector machines are used to develop emotion recognition systems with source, system, and prosodic features, respectively. Weighted combination of evidence has been used while combining the performance of systems developed using different features. From the results, it is observed that, each of the proposed speech features has contributed toward emotion recognition. The combination of features improved the emotion recognition performance, indicating the complementary nature of the features.

Emotion recognition from speech using source, system, and prosodic features

Automatic speech recognition (ASR) is a key technology in many services and applications. This typically requires user devices to send their speech data to the cloud for ASR decoding. As the speech signal carries a lot of information about the speaker, this raises serious privacy concerns. As a solution, an encoder may reside on each user device which performs local computations to anonymize the representation. In this paper, we focus on the protection of speaker identity and study the extent to which users can be recognized based on the encoded representation of their speech as obtained by a deep encoder-decoder architecture trained for ASR. Through speaker identification and verification experiments on the Librispeech corpus with open and closed sets of speakers, we show that the representations obtained from a standard architecture still carry a lot of information about speaker identity. We then propose to use adversarial training to learn representations that perform well in ASR while hiding speaker identity. Our results demonstrate that adversarial training dramatically reduces the closed-set classification accuracy, but this does not translate into increased open-set verification error hence into increased protection of the speaker identity in practice. We suggest several possible reasons behind this negative result.

Privacy-Preserving Adversarial Representation Learning in ASR: Reality or Illusion?

In the present study, we propose a regression-based scheme for the direct estimation of the height of unknown speakers from their speech. In this scheme every speech input is decomposed via the openSMILE audio parameterization to a single feature vector that is fed to a regression model, which provides a direct estimation of the persons’ height. The focus in this study is on the evaluation of the appropriateness of several linear and non-linear regression algorithms on the task of automatic height estimation from speech. The performance of the proposed scheme is evaluated on the TIMIT database, and the experimental results show an accuracy of 0.053 meters, in terms of mean absolute error, for the best performing Bagging regression algorithm. This accuracy corresponds to an averaged relative error of approximately 3%. We deem that the direct estimation of the height of unknown people from speech provides an important additional feature for improving the performance of various surveillance, profiling and access authorization applications.

Estimation of unknown speaker’s height from speech

A novel gender classification system has been proposed based on Gaussian Mixture Models, which apply the combined parameters of pitch and 10th order relative spectral perceptual linear predictive coefficients to model the characteristics of male and female speech The performances of gender classification system have been evaluated on the conditions of clean speech, noisy speech and multi-language The simulations show that the performance of the proposed gender classifier is excellent; it is very robust for noise and completely independent of languages; the classification accuracy is as high as above 98% for all clean speech and remains 95% for most noisy speech, even the SNR of speech is degraded to 0dB

/pdf/robust-gmm-based-gender-classification-using-pitch-and-rasta-xufeonelu3.pdf

Robust GMM Based Gender Classification using Pitch and RASTA-PLP Parameters of Speech

A modified AMDF pitch detection algorithm is presented. The amplitude character of the basic AMDF of speech signal is modified by means of a simple linear transform in our algorithm. Then the pitch-point determining method, that taking the global minimum valley point of AMDF as the pitch period calculation point, can be employed in pitch detection, even for the voiced speech with good stability and periodicity. The simulated experiments indicate the performance is improved considerably relative to the conventional AMDF method.

Modified AMDF pitch detection algorithm

This paper presents a new modified average magnitude difference function (AMDF) based on empirical mode decomposition (EMD) for pitch detection. We call it EMD-based AMDF (EMDAMDF). EMDAMDF inherits lots of advantages successfully from the conventional AMDF and eliminates the falling trend of the AMDF adaptively by means of EMD. Based on EMDAMDF, an effective pitch detection algorithm is proposed. The simulated results on Keele pitch reference database shows that the performance of the proposed EMDAMDF based pitch detection algorithm is obviously better than the original AMDF and its improvements (such as CAMDF and EAMDF) based algorithms.

Pitch detection using EMD-based AMDF

A robust classification system for children, adults male and adults female speech has been proposed. The proposed classifier is based on Gaussian Mixture Models, which apply the combined parameters of pitch, first three formants and 5-order relative spectral perceptual linear predictive coefficients (RASTA-PLPC) and Delta RASTA-PLPC, to model the characteristics of children speech, adult male and female speech. Its performances have been evaluated for several speech databases. The simulation results show that: for the speeches of adults and children in grades K to 5 (below the age of 11), the total accuracy of classification is above 97% in the condition of clean speech and still remains nearly 90% even if the SNR of speech is degraded to 5 dB in the condition of different additive noises and channel distortion noises. Furthermore, the performance of this classifier is independent of languages.

Robust Children and Adults Speech Classification

A novel text independent speaker identification system is proposed. In the proposed system, the 12-order perceptual linear predictive cepstrum and their delta coefficients in the span of five frames are extracted from segmented speech based on the method of pitch synchronous analysis. The Fisher ratio is used to evaluate the effectiveness of speech feature and select the part dimensions of the original 25-dimensional feature vector to form the new 13-dimensional feature vector. The Gaussian mixture model is applied to model the speakers. The experimental results show that the proposed system gives very good performances, which the identification accuracy is significantly better than that of the other 13-dimensional feature based systems and is a little bit better than or just the same as the 25-dimensional feature based system, but the algorithm complexity is much less than that of the 25-dimensional features based system.

Yumin Zeng

Papers

Robust GMM Based Gender Classification using Pitch and RASTA-PLP Parameters of Speech

Modified AMDF pitch detection algorithm

Pitch detection using EMD-based AMDF

Robust Children and Adults Speech Classification

Pitch Synchronous Analysis Method and Fisher Criterion Based Speaker Identification