Jean-Claude Martin

Bio: Jean-Claude Martin is an academic researcher from Université Paris-Saclay. The author has contributed to research in topics: Facial expression & Gesture. The author has an hindex of 21, co-authored 167 publications receiving 1634 citations. Previous affiliations of Jean-Claude Martin include Télécom ParisTech & Centre national de la recherche scientifique.

What Should a Generic Emotion Markup Language Be Able to Represent

Abstract: Working with emotion-related states in technological contexts requires a standard representation format. Based on that premise, the W3C Emotion Incubator group was created to lay the foundations for such a standard. The paper reports on two results of the group's work: a collection of use cases, and the resulting requirements. We compiled a rich collection of use cases, and grouped them into three types: data annotation, emotion recognition, and generation of emotion-related behaviour. Out of these, a structured set of requirements was distilled. It comprises the representation of the emotion-related state itself, some meta-information about that representation, various kinds of links to the "rest of the world", and several kinds of global metadata. We summarise the work, and provide pointers to the working documents containing full details.

The HUMAINE Database

Abstract: The HUMAINE Database is grounded in HUMAINE’s core emphasis on considering emotion in a broad sense – ‘pervasive emotion’ – and engaging with the way it colours action and interaction. The aim of the database is to provide a resource to which the community can go to see and hear the forms that emotion takes in everyday action and interaction, and to look at the tools that might be relevant to describing it. Earlier chapters in this handbook describe the techniques and models underpinning the collection and labelling of such data. This chapter focuses on conveying the range of forms that emotion takes in the database, the ways that they can be labelled and the issues that the data raises. The HUMAINE Database provides naturalistic clips which record that kind of material, in multiple modalities, and labelling techniques that are suited to describing it. It was clear when the HUMAINE project began that work on databases should form part of it. However there were very different directions that the work might have taken. They were encapsulated early on in the contrast between ‘supportive’ and ‘provocative’ approaches, introduced in an earlier chapter in this handbook. The supportive option was to assemble a body of data whose size and structure allowed it to be used directly to build systems for recognition and/or synthesis. The provocative option was to assemble a body of data that encapsulated the challenges that the field faces.

Multimodal indices to Japanese and French prosodically expressed social affects.

Abstract: Whereas several studies have explored the expression of emotions, little is known on how the visual and audio channels are combined during production of what we call the more controlled social affects, for example, "attitudinal" expressions. This article presents a perception study of the audovisual expression of 12 Japanese and 6 French attitudes in order to understand the contribution of audio and visual modalities for affective communication. The relative importance of each modality in the perceptual decoding of the expressions of four speakers is analyzed as a first step towards a deeper comprehension of their influence on the expression of social affects. Then, the audovisual productions of two speakers (one for each language) are acoustically (F0, duration and intensity) and visually (in terms of Action Units) analyzed, in order to match the relation between objective parameters and listeners' perception of these social affects. The most pertinent objective features, either acoustic or visual, are then discussed, in a bilingual perspective: for example, the relative influence of fundamental frequency for attitudinal expression in both languages is discussed, and the importance of a certain aspect of the voice quality dimension in Japanese is underlined.

Collection and Annotation of a Corpus of Human-Human Multimodal Interactions: Emotion and Others Anthropomorphic Characteristics

Abstract: In order to design affective interactive systems, experimental grounding is required for studying expressions of emotion during interaction. In this paper, we present the EmoTaboo protocol for the collection of multimodal emotional behaviours occurring during human-human interactions in a game context. First annotations revealed that the collected data contains various multimodal expressions of emotions and other mental states. In order to reduce the influence of language via a predetermined set of labels and to take into account differences between coders in their capacity to verbalize their perception, we introduce a new annotation methodology based on 1) a hierarchical taxonomy of emotion-related words, and 2) the design of the annotation interface. Future directions include the implementation of such an annotation tool and its evaluation for the annotation of multimodal interactive and emotional behaviours. We will also extend our first annotation scheme to several other characteristics interdependent of emotions.

IEMOCAP: interactive emotional dyadic motion capture database

Abstract: Since emotions are expressed through a combination of verbal and non-verbal channels, a joint analysis of speech and gestures is required to understand expressive human communication. To facilitate such investigations, this paper describes a new corpus named the “interactive emotional dyadic motion capture database” (IEMOCAP), collected by the Speech Analysis and Interpretation Laboratory (SAIL) at the University of Southern California (USC). This database was recorded from ten actors in dyadic sessions with markers on the face, head, and hands, which provide detailed information about their facial expressions and hand movements during scripted and spontaneous spoken communication scenarios. The actors performed selected emotional scripts and also improvised hypothetical scenarios designed to elicit specific types of emotions (happiness, anger, sadness, frustration and neutral state). The corpus contains approximately 12 h of data. The detailed motion capture information, the interactive setting to elicit authentic emotions, and the size of the database make this corpus a valuable addition to the existing databases in the community for the study and modeling of multimodal and expressive human communication.