Data Mining Practical Machine Learning Tools and Techniques

Emotion and Adaptation

Since emotions are expressed through a combination of verbal and non-verbal channels, a joint analysis of speech and gestures is required to understand expressive human communication. To facilitate such investigations, this paper describes a new corpus named the “interactive emotional dyadic motion capture database” (IEMOCAP), collected by the Speech Analysis and Interpretation Laboratory (SAIL) at the University of Southern California (USC). This database was recorded from ten actors in dyadic sessions with markers on the face, head, and hands, which provide detailed information about their facial expressions and hand movements during scripted and spontaneous spoken communication scenarios. The actors performed selected emotional scripts and also improvised hypothetical scenarios designed to elicit specific types of emotions (happiness, anger, sadness, frustration and neutral state). The corpus contains approximately 12 h of data. The detailed motion capture information, the interactive setting to elicit authentic emotions, and the size of the database make this corpus a valuable addition to the existing databases in the community for the study and modeling of multimodal and expressive human communication.

/pdf/iemocap-interactive-emotional-dyadic-motion-capture-database-2ffyt1sb5i.pdf

IEMOCAP: interactive emotional dyadic motion capture database

The cultural origins of human cognition

Creativity: Flow and the Psychology of Discovery and Invention

Recent technologies enable the exploitation of full body expressions in applications such as interactive arts but are still limited in terms of dyadic subtle interaction patterns. Our project aims at full body expressive interactions between a user and an autonomous virtual agent. The currently available databases do not contain full body expressivity and interaction patterns via avatars. In this paper, we describe a protocol defined to collect a database to study expressive full-body dyadic interactions. We detail the coding scheme for manually annotating the collected videos. Reliability measures for global annotations of expressivity and interaction are also provided.

/pdf/a-database-of-full-body-virtual-interactions-annotated-with-ol3iz6n9ls.pdf

A Database of Full Body Virtual Interactions Annotated with Expressivity Scores

This study investigates a specific aspect of joint attention, that is, the emergence of the sense that one is leading the attentional focus of others. Thirty participants were placed in front of two avatars and had to pay attention to objects that were also attended to by the avatars. Unbeknownst to the participant, the avatars’ gaze orientations were alternately controlled by the participant’s eyes. Eye-tracking data were collected and participants were enquired about their experience to account for their sense of agency. Only five participants noticed that the avatars were following their gaze. The 25 participants who remained unaware that they controlled the avatars nevertheless modified their gazing behavior with an increase in direct gaze toward one of the avatars. This outcome highlights a dissociation between an implicit feeling that a change occurred and the explicit awareness of being the cause of this change.

The awareness of joint attention: A study using gaze contingent avatars

VDL+LEA: complémentarité entre interaction multi-modale et interaction multi-agents

Relations between emotions and multimodal behaviors have mostly been studied in the case of acted basic emotions. In this paper, we describe two experiments studying these relations with a copy-synthesis approach. We start from video clips of TV interviews including real-life behaviors. A protocol and a coding scheme have been defined for annotating these clips at several levels (context, emotion, multimodality). The first experiment enabled to manually identify the levels of representation required for replaying the annotated behaviors by an expressive agent. The second experiment involved automatic extraction of information from the multimodal annotations. Such an approach enables to study the complex relations between emotion and multimodal behaviors.

Utilisation des différents niveaux de représentation multimodale et émotionnelle

Pour revetir un substrat (5) d'une couche mince (6) d'oxy de de platine pur, substance isolante et transparente, on a recours a un magnetron cylindrique comprenant des moyens (4) pour engendrer un champ magnetique axial et, dans une enceinte (1) contenant de l'oxygene rarefie, une cathode axiale (2) au moins revetue de platine et une grille cylindrique (3) for mant anode et entourant coaxialement la cathode de facon a reserver une zone annulaire B a l'exterieur de la grille et l'on place le substrat dans cette zone B de facon telle que sa face a revetir par la couche mince soit a l'abri de l'impact des electrons. Cette mise a l'abri peut egalement etre obtenue par une forme particuliere donnee aux barreaux constituant la grille ou par la mise en place d'obstacles dans la zone B. Il suffit ensuite de chauffer la couche mince, localement ou non, a plus de 250°C, pour la transformer en platine conducteur et opa que.

Jean-Claude Martin

Papers

A Database of Full Body Virtual Interactions Annotated with Expressivity Scores

The awareness of joint attention: A study using gaze contingent avatars

VDL+LEA: complémentarité entre interaction multi-modale et interaction multi-agents

Utilisation des différents niveaux de représentation multimodale et émotionnelle

Substrates thinly coated with platinum oxide, devices for obtaining the so coated substrates and products obtained from substrates so coated