語用論(Pragmatics)を考える

Automated analysis of human affective behavior has attracted increasing attention from researchers in psychology, computer science, linguistics, neuroscience, and related disciplines. However, the existing methods typically handle only deliberately displayed and exaggerated expressions of prototypical emotions despite the fact that deliberate behaviour differs in visual appearance, audio profile, and timing from spontaneously occurring behaviour. To address this problem, efforts to develop algorithms that can process naturally occurring human affective behaviour have recently emerged. Moreover, an increasing number of efforts are reported toward multimodal fusion for human affect analysis including audiovisual fusion, linguistic and paralinguistic fusion, and multi-cue visual fusion based on facial expressions, head movements, and body gestures. This paper introduces and surveys these recent advances. We first discuss human emotion perception from a psychological perspective. Next we examine available approaches to solving the problem of machine understanding of human affective behavior, and discuss important issues like the collection and availability of training and test data. We finally outline some of the scientific and engineering challenges to advancing human affect sensing technology.

A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions

Survey on speech emotion recognition: Features, classification schemes, and databases

This survey describes recent progress in the field of Affective Computing (AC), with a focus on affect detection. Although many AC researchers have traditionally attempted to remain agnostic to the different emotion theories proposed by psychologists, the affective technologies being developed are rife with theoretical assumptions that impact their effectiveness. Hence, an informed and integrated examination of emotion theories from multiple areas will need to become part of computing practice if truly effective real-world systems are to be achieved. This survey discusses theoretical perspectives that view emotions as expressions, embodiments, outcomes of cognitive appraisal, social constructs, products of neural circuitry, and psychological interpretations of basic feelings. It provides meta-analyses on existing reviews of affect detection systems that focus on traditional affect detection modalities like physiology, face, and voice, and also reviews emerging research on more novel channels such as text, body language, and complex multimodal systems. This survey explicitly explores the multidisciplinary foundation that underlies all AC applications by describing how AC researchers have incorporated psychological theories of emotion and how these theories affect research questions, methods, results, and their interpretations. In this way, models and methods can be compared, and emerging insights from various disciplines can be more expertly integrated.

/pdf/affect-detection-an-interdisciplinary-review-of-models-49pcvdc6da.pdf

Affect Detection: An Interdisciplinary Review of Models, Methods, and Their Applications

This article is a survey of methods for measuring agreement among corpus annotators. It exposes the mathematics and underlying assumptions of agreement coefficients, covering Krippendorff's alpha as well as Scott's pi and Cohen's kappa; discusses the use of coefficients in several annotation tasks; and argues that weighted, alpha-like coefficients, traditionally less used than kappa-like measures in computational linguistics, may be more appropriate for many corpus annotation tasks---but that their use makes the interpretation of the value of the coefficient even harder.

/pdf/inter-coder-agreement-for-computational-linguistics-2mpuyn1mc8.pdf

Inter-coder agreement for computational linguistics

We investigate the use of prosody for the detection of frustr ation and annoyance in natural human-computer dialog. In addition to prosodic features, we examine the contribution of language model information and speaking “style”. Results show that a prosodic model can predict whether an utterance is neutral ve sus “annoyed or frustrated” with an accuracy on par with that of human interlabeler agreement. Accuracy increases when dis criminating only “frustrated” from other utterances, and when u sing only those utterances on which labelers originally agreed. Furthermore, prosodic model accuracy degrades only slightly when u si g recognized versus true words. Language model features, eve n if based on true words, are relatively poor predictors of frust ration. Finally, we find that hyperarticulation is not a good predict or of emotion; the two phenomena often occur independently.

Prosody-based automatic detection of annoyance and frustration in human-computer dialog.

: We describe a new corpus of over 180,000 hand- annotated dialog act tags and accompanying adjacency pair annotations for roughly 72 hours of speech from 75 naturally-occurring meetings. We provide a brief summary of the annotation system and labeling procedure, inter-annotator reliability statistics, overall distributional statistics, a description of auxiliary files distributed with the corpus, and information on how to obtain the data.

/pdf/the-icsi-meeting-recorder-dialog-act-mrda-corpus-rkh4w8imje.pdf

The ICSI Meeting Recorder Dialog Act (MRDA) Corpus

We explore the two related tasks of dialog act (DA) segmentation and DA classification for speech from the ICSI Meeting Corpus. We employ simple lexical and prosodic knowledge sources, and compare results for human-transcribed versus automatically recognized words. Since there is little previous work on DA segmentation and classification in the meeting domain, our study provides baseline performance rates for both tasks. We introduce a range of metrics for use in evaluation, each of which measures different aspects of interest. Results show that both tasks are difficult, particularly for a fully automatic system. We find that a very simple prosodic model aids performance over lexical information alone, especially for segmentation. Both tasks, but particularly word-based segmentation, are degraded by word recognition errors. Finally, while classification results for meeting data show some similarities to previous results for telephone conversations, findings also suggest a potential difference with respect to the effect of modeling DA context.

http://http.icsi.berkeley.edu/ftp/global/pub/speech/papers/icassp2005-da-seg-class.pdf

Automatic dialog act segmentation and classification in multiparty meetings

This paper provides a progress report on ICSI s Meeting Project, including both the data collected and annotated as part of the pro-ject, as well as the research lines such materials support. We include a general description of the official ICSI Meeting Corpus , as currently available through the Linguistic Data Consortium, discuss some of the existing and planned annotations which augment the basic transcripts provided there, and describe several research efforts that make use of these materials. The corpus supports wide-ranging efforts, from low-level processing of the audio signal (including automatic speech transcription, speaker tracking, and work on far-field acoustics) to higher-level analyses of meeting structure, content, and interactions (such as topic and sentence segmentation, and automatic detection of dialogue acts and meeting hot spots ).

http://http.icsi.berkeley.edu/ftp/pub/speech/papers/nist2004-meeting-janin.pdf

The ICSI Meeting Project: Resources and Research

Both human and automatic processing of speech require recognition of more than just words. In this paper we provide a brief overview of research on structural metadata extraction in the DARPA EARS rich transcription program. Tasks include detection of sentence boundaries, filler words, and disfluencies. Modeling approaches combine lexical, prosodic, and syntactic information, using various modeling techniques for knowledge source integration. The performance of these methods is evaluated by task, by data source (broadcast news versus spontaneous telephone conversations) and by whether transcriptions come from humans or from an (errorful) automatic speech recognizer. A representative sample of results shows that combining multiple knowledge sources (words, prosody, syntactic information) is helpful, that prosody is more helpful for news speech than for conversational speech, that word errors significantly impact performance, and that discriminative models generally provide benefit over maximum likelihood models. Important remaining issues, both technical and programmatic, are also discussed.

/pdf/structural-metadata-research-in-the-ears-program-hm7717teu0.pdf

J. Ang

Papers

Prosody-based automatic detection of annoyance and frustration in human-computer dialog.

The ICSI Meeting Recorder Dialog Act (MRDA) Corpus

Automatic dialog act segmentation and classification in multiparty meetings

The ICSI Meeting Project: Resources and Research

Structural metadata research in the EARS program