https://dspace.stir.ac.uk/bitstream/1893/25490/1/affective-computing-review.pdf

A review of affective computing

Affect detection is an important pattern recognition problem that has inspired researchers from several areas The field is in need of a systematic review due to the recent influx of Multimodal (MM) affect detection systems that differ in several respects and sometimes yield incompatible results This article provides such a survey via a quantitative review and meta-analysis of 90 peer-reviewed MM systems The review indicated that the state of the art mainly consists of person-dependent models (622p of systems) that fuse audio and visual (556p) information to detect acted (522p) expressions of basic emotions and simple dimensions of arousal and valence (645p) with feature- (389p) and decision-level (356p) fusion techniques However, there were also person-independent systems that considered additional modalities to detect nonbasic emotions and complex dimensions using model-level fusion techniques The meta-analysis revealed that MM systems were consistently (85p of systems) more accurate than their best unimodal counterparts, with an average improvement of 983p (median of 660p) However, improvements were three times lower when systems were trained on natural (459p) versus acted data (127p) Importantly, MM accuracy could be accurately predicted (cross-validated R2 of 0803) from unimodal accuracies and two system-level factors Theoretical and applied implications and recommendations are discussed

A Review and Meta-Analysis of Multimodal Affect Detection Systems

Considerable progress has been made in handwriting recognition technology over the last few years. Thus far, handwriting recognition systems have been limited to small and medium vocabulary applications, since most of them often rely on a lexicon during the recognition process. The capability of dealing with large lexicons, however, opens up many more applications. This article will discuss the methods and principles that have been proposed to handle large vocabularies and identify the key issues affecting their future deployment. To illustrate some of the points raised, a large vocabulary off-line handwritten word recognition system will be described.

/pdf/large-vocabulary-off-line-handwriting-recognition-a-survey-2rmehscx29.pdf

Large vocabulary off-line handwriting recognition: A survey

An alternative approach to speaker adaptation for a large-vocabulary hidden-Markov-model-based speech recognition system is described. The goal of this investigation was to train the IBM speech recognition system with only five minutes of speech data from a new speaker instead of the usual 20 minutes without the recognition rate dropping by more than 1-2%. The approach is based on the use of a stochastic model representing the different properties of the new speaker and an old speaker for which the full training set of 20 minutes is available. It is called a speaker Markov model. It is shown how the parameters of such a model can be derived and how it can be used for transforming the training set of the old speaker in order to use it in addition to the short training set of the new speaker. The adaptation algorithm was tested with 12 speakers. The average recognition rate dropped from 96.4% to 95.2% for a 5000-word vocabulary task. The decoding time increased by a factor of 1.35; this factor is often 3-5 if other adaptation algorithms are used. >

Speaker adaptation for large vocabulary speech recognition systems using speaker Markov models

The automatic estimation of human affect from the speech signal is an important step towards making virtual agents more natural and human-like. In this paper, we present a novel technique for incremental recognition of the user's emotional state as it is applied in a sensitive artificial listener (SAL) system designed for socially competent human-machine communication. Our method is capable of using acoustic, linguistic, as well as long-range contextual information in order to continuously predict the current quadrant in a two-dimensional emotional space spanned by the dimensions valence and activation. The main system components are a hierarchical dynamic Bayesian network (DBN) for detecting linguistic keyword features and long short-term memory (LSTM) recurrent neural networks which model phoneme context and emotional history to predict the affective state of the user. Experimental evaluations on the SAL corpus of non-prototypical real-life emotional speech data consider a number of variants of our recognition framework: continuous emotion estimation from low-level feature frames is evaluated as a new alternative to the common approach of computing statistical functionals of given speech turns. Further performance gains are achieved by discriminatively training LSTM networks and by using bidirectional context information, leading to a quadrant prediction F1-measure of up to 51.3 %, which is only 7.6 % below the average inter-labeler consistency.

/pdf/combining-long-short-term-memory-and-dynamic-bayesian-5gum2hsmvr.pdf

Combining Long Short-Term Memory and Dynamic Bayesian Networks for Incremental Emotion-Sensitive Artificial Listening

A variety of approaches towards Speech Emotion Recognition were presented since the research activities started in the late last decade. Today we are all aware of the great importance of emotional aspects as the next step towards more natural human-machine interaction. Yet, a growing number of isolated individual groups contribute to the general advances in the exciting landscape we see today. However, there is still a strong need for international cross-fertilization on the one hand in view of signal pre-processing, feature selection, and classification methods, and furthermore comparability with respect to emotion classes, and databases. As first commercial products arise today, the research community is challenged by more realistic out-of-the-lab conditions. This leads to the next level of Speech Emotion Recognition aiming among others at a robust recognition, independent of the speaker, backgroundnoise, and to a certain extent even of the spoken language by reasonable integration of acoustic, linguistic, and context information, as well as spotting for emotions out of a stream. In this article we like to give an overview of the existing pieces of the puzzle, introduce novel impulses, and aim to investigate future directions in this young discipline.

Speech Emotion Recognition Exploiting Acoustic and Linguistic Information Sources

In this paper a robust Script recognition system is described, which makes use of a language model, that consists of backoff character n-grams. The system is based on Hidden Markov Models (HMMs) using discrete and hybrid modeling techniques, where the latter depends on a vector quantizer trained according to the MMI-criterion (information theory-based neural network). The presented recognition results refer to the SEDAL-database of degraded English documents such as photocopy or fax using no dictionary and a writer-dependent handwritten database of cursive German Script samples. Our resulting system for character recognition yields significantly better recognition results for an unlimited vocabulary using language models.

Unlimited Vocabulary Script Recognition Using Character N-Grams

This paper presents a brief description of the ALERT system, which is under development by a consortium working on a research project sponsored by the European Commission. The ALERT system uses advanced speech recognition technology and video processing techniques in order to process large broadcast speech archives and multimedia information resources for the purpose of extracting specific information from such databases and inform selected customers about its contents. It is one of the most ambitious projects currently carried out in the human language technologies (HLT) area (see also http://alert.uni-duisburg.de). The paper describes the objectives of the overall system, its basic system architecture and the scientific approach taken in order to realize the specified demonstrators.

The ALERT system: advanced broadcast speech recognition technology for selective dissemination of multimedia information

In this contribution we like to give a very short introduction to a system allowing for enhanced robustness in emotion recognition from speech. The underlying emotion set consists in seven discrete states derived from the MPEG-4 standard: Anger, disgust, fear, joy, neutral, sadness, and surprise. Within here several novel approaches and investigations to acoustic feature sets, semantic analysis of spoken content, and the combination of those two information streams in a soft decision fusion are presented in short. Firstly statistic features on prosody, extracted from the speech signal, are ranked by their quantitative contribution to the estimation of an emotion. After investigation of various classification methods Support Vector Machines performed out within this task. Secondly an approach to emotion recognition by the spoken content is introduced applying Bayesian Network based spotting for emotional key-phrases. Finally the two information sources will be integrated in a soft decision fusion by using a MLP, which eventually leads to a remarkable improvement in overall recognition results.

Enhanced Robustness in Speech Emotion Recognition Combining Acoustic and Semantic Analyses

http://link.springer.com/content/pdf/10.1007/978-94-009-3117-6_26.pdf

G. Rigoll

Papers

Speech Emotion Recognition Exploiting Acoustic and Linguistic Information Sources

Unlimited Vocabulary Script Recognition Using Character N-Grams

The ALERT system: advanced broadcast speech recognition technology for selective dissemination of multimedia information

Enhanced Robustness in Speech Emotion Recognition Combining Acoustic and Semantic Analyses

Experiments With the Acoustic Processor of A Hidden Markov Model-Based, Large Vocabulary Speech Recognition System