Data Mining Practical Machine Learning Tools and Techniques

Emotion and Adaptation

Since emotions are expressed through a combination of verbal and non-verbal channels, a joint analysis of speech and gestures is required to understand expressive human communication. To facilitate such investigations, this paper describes a new corpus named the “interactive emotional dyadic motion capture database” (IEMOCAP), collected by the Speech Analysis and Interpretation Laboratory (SAIL) at the University of Southern California (USC). This database was recorded from ten actors in dyadic sessions with markers on the face, head, and hands, which provide detailed information about their facial expressions and hand movements during scripted and spontaneous spoken communication scenarios. The actors performed selected emotional scripts and also improvised hypothetical scenarios designed to elicit specific types of emotions (happiness, anger, sadness, frustration and neutral state). The corpus contains approximately 12 h of data. The detailed motion capture information, the interactive setting to elicit authentic emotions, and the size of the database make this corpus a valuable addition to the existing databases in the community for the study and modeling of multimodal and expressive human communication.

/pdf/iemocap-interactive-emotional-dyadic-motion-capture-database-2ffyt1sb5i.pdf

IEMOCAP: interactive emotional dyadic motion capture database

The cultural origins of human cognition

Creativity: Flow and the Psychology of Discovery and Invention

Affective computing is among emerging fields for the design of new Human-Computer Interfaces. The facial expression was considered as the main way to communicate emotions. However, this approach presents some limitations for the recognition of complex emotions. Recent researches highlight new approaches to communicate effectively some emotions. The use of haptic feedback seems to be a promising solution to improve the level of recognition. In fact, haptic feedback, introduces different physical dimensions which complete the visual features of emotions. This paper proposes the study of the complementary between facial expressions and haptic feedback to recognize different class of emotions. Results show that the proposed haptic feedback improves at different levels the recognition of some emotions.

Improvement of the recognition of facial expressions with haptic feedback

We demonstrate how the Tycoon framework can be put to practice with the Anvil tool in a concrete case study. Tycoon offers a coding scheme and analysis metrics for multimodal communication scenarios. Anvil is a generic, extensible and ergonomically designed annotation tool for videos. In this paper, we describe the Anvil tool, the Tycoon scheme/metrics, and their implementation in Anvil for a video sample. A new Anvil feature, motivated by the Tycoon scheme, is presented: non-temporal annotation objects – an important concept, we argue, of general interest. We also outline future plans for automatizing Tycoon metrics computation using Anvil plug-ins.

/pdf/annotating-and-measuring-multimodal-behaviour-tycoon-metrics-1za958upyr.pdf

Annotating and Measuring Multimodal Behaviour – Tycoon Metrics in the Anvil Tool

Recent research suggests that predicting students' emotions during e-learning is quite relevant but should be situated in the learning context and consider the individual profile of users. More knowledge is required for assessing the possible contributions of multiple sources of information for predicting students' emotions. In this paper we describe an ontology that we have implemented for predicting students' emotions when interacting with a quiz about Java programming. An experimental study with 17 computer science students compares the automatic predictions made by the ontology with the emotions self-reported by students.

An ontology for predicting students' emotions during a quiz. Comparison with self-reported emotions

Imitation is of major importance during social interactions, would it be between humans or between a human and a robot. This is even more true when considering users with special needs. In this paper, we describe an experimental imitation task protocol using a robot Nao that we designed to assess whether sensory profiles of children with Autistic Spectrum Disorder (ASD) influence their capabilities to imitate or to initiate gestures that are going to be imitated. We based our work on the hypothesis that children with an overreliance on proprioceptive cues and hyporeactivity to visual cues have a greater difficulty imitating and improve their skills more slowly than children with an overreliance on visual cues and hyporeactivity to proprioceptive cues. Twelve children and teenagers with ASD participated in seven imitation sessions over eight weeks. As expected, we observed that children with an overreliance on proprioceptive cues and hyporeactivity to visual cues had more difficulties imitating the robot than the other children. Moreover, the repeated sessions revealed to have positive effects on social behaviors displayed by all children (gaze to the partner, imitations) toward a human partner after the sessions with the robot. We conclude on the possible impacts of such results on the design of social human-robot interactions for users with ASD.

Do Sensory Preferences of Children with Autism Impact an Imitation Task with a Robot

Consistency and coherence in behaviors are key concepts for creating believable virtual agents. Personality is described by psychology as a set of stable and individualized behaviors. Thus, endowing agents with a personality can help agents to exhibit consistent behaviors. In this paper, we suggest using a socio-cognitive perspective to build a model of personality for virtual agents and to highlight some of the personality structures and processes. Because many researchers in personality psychology emphasize the role of the self in personality, we propose PERSEED, a conceptual model of personality that is based on self-regulation theories.

Jean-Claude Martin

Papers

Improvement of the recognition of facial expressions with haptic feedback

Annotating and Measuring Multimodal Behaviour – Tycoon Metrics in the Anvil Tool

An ontology for predicting students' emotions during a quiz. Comparison with self-reported emotions

Do Sensory Preferences of Children with Autism Impact an Imitation Task with a Robot

PERSEED: A Self-Based Model of Personality for Virtual Agents Inspired by Socio-cognitive Theories