http://www.cs.columbia.edu/~julia/papers/Scherer03.pdf

Vocal communication of emotion: a review of research paradigms

http://www5.informatik.uni-erlangen.de/Forschung/Publikationen/2011/Schuller11-RRE.pdf

Recognising realistic emotions and affect in speech: State of the art and lessons learnt from the first challenge

From Discourse to Logic

We present SimSensei Kiosk, an implemented virtual human interviewer designed to create an engaging face-to-face interaction where the user feels comfortable talking and sharing information. SimSensei Kiosk is also designed to create interactional situations favorable to the automatic assessment of distress indicators, defined as verbal and nonverbal behaviors correlated with depression, anxiety or post-traumatic stress disorder (PTSD). In this paper, we summarize the design methodology, performed over the past two years, which is based on three main development cycles: (1) analysis of face-to-face human interactions to identify potential distress indicators, dialogue policies and virtual human gestures, (2) development and analysis of a Wizard-of-Oz prototype system where two human operators were deciding the spoken and gestural responses, and (3) development of a fully automatic virtual interviewer able to engage users in 15-25 minute interactions. We show the potential of our fully automatic virtual human interviewer in a user study, and situate its performance in relation to the Wizard-of-Oz prototype.

/pdf/simsensei-kiosk-a-virtual-human-interviewer-for-healthcare-18t9klo36e.pdf

SimSensei kiosk: a virtual human interviewer for healthcare decision support

This paper describes an international effort to unify a multimodal behavior generation framework for Embodied Conversational Agents (ECAs) We propose a three stage model we call SAIBA where the stages represent intent planning, behavior planning and behavior realization A Function Markup Language (FML), describing intent without referring to physical behavior, mediates between the first two stages and a Behavior Markup Language (BML) describing desired physical realization, mediates between the last two stages In this paper we will focus on BML The hope is that this abstraction and modularization will help ECA researchers pool their resources to build more sophisticated virtual humans

/pdf/towards-a-common-framework-for-multimodal-generation-the-39et4d6nb1.pdf

Towards a common framework for multimodal generation: the behavior markup language

This paper describes an international effort to unify a multimodal behavior generation framework for Embodied Conversational Agents (ECAs). We propose a three stage model we call SAIBA where the stages represent intent planning, behavior planning and behavior realization. A Function Markup Language (FML), describing intent without referring to physical behavior, mediates between the first two stages and a Behavior Markup Language (BML) describing desired physical realization, mediates between the last two stages. In this paper we will focus on BML. The hope is that this ion and modularization will help ECA researchers pool their resources to build more sophisticated virtual humans.

Towards a common framework for multimodal generation : The behavior markup language

In this paper, we describe the Rich Representation Language (RRL) which is used in the NECA system. The NECA system generates interactions between two or more animated characters. The RRL is a formal framework for representing the information that is exchanged at the interfaces between the various NECA system modules.

/pdf/rrl-a-rich-representation-language-for-the-description-of-3r4v5e0h1u.pdf

RRL: A Rich Representation Language for the Description of Agent Behaviour in NECA

We describe the attempt to synthesize emotional speech with a concatenative speech synthesizer using a parameter space covering not only f0, duration and amplitude, but also voice quality parameters, spectral energy distribution, harmonics-to-noise ratio, and articulatory precision. The application of these extended parameter set offers the possibility to combine the high segmental quality of concatenative synthesis with a wider range of control settings needed for the synthesis of natural affected speech.

Generating emotional speech with a concatenative synthesizer.

Working with emotion-related states in technological contexts requires a standard representation format. Based on that premise, the W3C Emotion Incubator group was created to lay the foundations for such a standard. The paper reports on two results of the group's work: a collection of use cases, and the resulting requirements. We compiled a rich collection of use cases, and grouped them into three types: data annotation, emotion recognition, and generation of emotion-related behaviour. Out of these, a structured set of requirements was distilled. It comprises the representation of the emotion-related state itself, some meta-information about that representation, various kinds of links to the "rest of the world", and several kinds of global metadata. We summarise the work, and provide pointers to the working documents containing full details.

/pdf/what-should-a-generic-emotion-markup-language-be-able-to-3iykvr3abg.pdf

Hannes Pirker

Papers

Towards a common framework for multimodal generation: the behavior markup language

Towards a common framework for multimodal generation : The behavior markup language

RRL: A Rich Representation Language for the Description of Agent Behaviour in NECA

Generating emotional speech with a concatenative synthesizer.

What Should a Generic Emotion Markup Language Be Able to Represent