Aggression recognition using overlapping speech
01 Oct 2017-pp 299-304
TL;DR: The findings show that overlapping speech is a key feature for predicting aggression levels, that discriminating only severe cases of overlap is a sufficient feature and that automatically predicted overlap is improving aggression recognition as well.
Abstract: Automatic recognition of negative affect and aggression is key in many safety critical domains such as surveillance and health care In this paper we explore the potential of overlapping speech for predicting aggression levels As a first step we consider 3 categories of overlapping speech based on literature Having an annotation of these overlap categories, we examine whether overlapping speech is a good feature for predicting aggression by using it in classification together with a set of acoustic features typically used for this purpose Next, we explore if this fine categorization of overlap is necessary in predicting aggression levels or a more coarse representation is sufficient Finally, we check the additive values of automatically predicted overlapping speech for aggression recognition The experiments are performed on a dataset of dyadic interactions between professional aggression training actors (actors) and naive participants (students) interacting freely based on short role descriptions Our findings show that overlapping speech is a key feature for predicting aggression levels, that discriminating only severe cases of overlap is a sufficient feature and that automatically predicted overlap is improving aggression recognition as well
Citations
More filters
18 Oct 2021
TL;DR: In this article, the authors presented a series of experiments to automatically recognize interdependence perceptions in dyadic face-to-face negotiations using these sources, and they demonstrated that differences in some types of interdependent perceptions can be detected through the automatic analysis of nonverbal behaviors.
Abstract: Enabling computer-based applications to display intelligent behavior in complex social settings requires them to relate to important aspects of how humans experience and understand such situations. One crucial driver of peoples’ social behavior during an interaction is the interdependence they perceive, i.e., how the outcome of an interaction is determined by their own and others’ actions. According to psychological studies, both the nonverbal behavior displayed by Motivated by this, we present a series of experiments to automatically recognize interdependence perceptions in dyadic face-to-face negotiations using these sources. Concretely, our approach draws on a combination of features describing individuals’ Facial, Upper Body, and Vocal Behavior with state-of-the-art algorithms for multivariate time series classification. Our findings demonstrate that differences in some types of interdependence perceptions can be detected through the automatic analysis of nonverbal behaviors. We discuss implications for developing socially intelligent systems and opportunities for future research.
7 citations
01 Jan 2019
TL;DR: This work argues that, in this context, big data alone is not purposeful, since important effects are obscured, and since high-quality annotation is too costly, and encourages the collection and use of enriched data.
Abstract: Contemporary technical devices obey the paradigm of naturalistic multimodal interaction and user-centric individualisation. Users expect devices to interact intelligently, to anticipate their needs, and to adapt to their behaviour. To do so, companion-like solutions have to take into account the affective and dispositional state of the user, and therefore to be trained and modified using interaction data and corpora. We argue that, in this context, big data alone is not purposeful, since important effects are obscured, and since high-quality annotation is too costly. We encourage the collection and use of enriched data. We report on recent trends in this field, presenting methodologies for collecting data with rich disposition variety and predictable classifications based on a careful design and standardised psychological assessments. Besides socio-demographic information and personality traits, we also use speech events to improve user state models. Furthermore, we present possibilities to increase the amount of enriched data in cross-corpus or intra-corpus way based on recent learning approaches. Finally, we highlight particular recent neural recognition approaches feasible for smaller datasets, and covering temporal aspects.
6 citations
TL;DR: In this article , the authors investigated dyadic human-human interactions and focused on the relations between the emotional changes occurring around overlaps in both interaction participants, and presented a classification approach based on features derived from such emotional changes surrounding an overlap and compared the classification performance of these features to classic acoustic features.
Abstract: Although being a frequently occurring phenomenon in spoken communication, speech overlaps did not obtain the deserved attention in research so far—in both Human-Human Interaction (HHI) and Human-Computer Interaction (HCI). It is common knowledge that overlaps can figure as a competitive, rude interruption as well as a cooperative, convenient feedback signal giving important insight on the course of the interaction—but how are they related to the internal state of the overlapping speaker or the overlapped speaker? In this paper, we investigate dyadic human-human interactions and focus on the relations between the emotional changes occurring around overlaps in both interaction participants. Further to an in-depth statistical analysis of the changes in control and valence levels with respect to the nature of the overlap, we also present a classification approach based on features derived from such emotional changes surrounding an overlap and compare the classification performance of these features to classic acoustic features. We show that the automatic classification of competitive and cooperative overlaps using the changes in valence and control levels of the overlapping speaker outperforms common approaches employing acoustic and linguistic features.
1 citations
TL;DR: This work proposes a Semi-Supervised Learning (SSL) method for cross-lingual emotion recognition when a few labels from the new language are available and adapts to a new language by exploiting a pseudo-labeling strategy for the unlabeled utterances.
Abstract: Speech emotion recognition (SER) on a single language has achieved remark-able results through deep learning approaches over the last decade. However, cross-lingual SER remains a challenge in real-world applications due to (i) a large difference between the source and target domain distributions, (ii) the availability of few labeled and many unlabeled utterances for the new language. Taking into account previous aspects, we propose a Semi-Supervised Learning (SSL) method for cross-lingual emotion recognition when a few labels from the new language are available. Based on a Convolutional Neural Network (CNN), our method adapts to a new language by exploiting a pseudo-labeling strategy for the unlabeled utterances. In particular, the use of a hard and soft pseudo-labels approach is investigated. We thoroughly evaluate the performance of the method in a speaker-independent setup on both the source and the new language and show its robustness across five languages belonging to different linguistic strains.
1 citations
TL;DR: In this paper , the means of verbal aggression expression as used in the speech of female and male native speakers of English in corporate communication within the gender framework in the philosophy of culture is analyzed using the transcripts of talks and business meetings to single out their common use patterns.
Abstract: The article sheds light on the means of verbal aggression expression as used in the speech of female and male native speakers of English in corporate communication within the gender framework in the philosophy of culture. The means of expression are analysed using the transcripts of talks and business meetings to single out their common use patterns. Research methodology is premised on the philosophical approach to culture, statistical data analysis, methods of continuous sampling, definitional analysis, textual analysis, component analysis and complex analysis of vocabulary units, as well as lexical and stylistic analysis. Upon examining the peculiarities of speech aggression, the authors address gender differences in speech act production. The study demonstrates the significance of the gender factor in implementing verbal behaviour strategies in cases of verbal aggression within corporate communication while also revealing some significant differences in male and female speech aggression in corporate communication (negative connotations in women’s and men’s speech).
1 citations
References
More filters
TL;DR: This paper provides an introduction to the WEKA workbench, reviews the history of the project, and, in light of the recent 3.6 stable release, briefly discusses what has been added since the last stable version (Weka 3.4) released in 2003.
Abstract: More than twelve years have elapsed since the first public release of WEKA. In that time, the software has been rewritten entirely from scratch, evolved substantially and now accompanies a text on data mining [35]. These days, WEKA enjoys widespread acceptance in both academia and business, has an active community, and has been downloaded more than 1.4 million times since being placed on Source-Forge in April 2000. This paper provides an introduction to the WEKA workbench, reviews the history of the project, and, in light of the recent 3.6 stable release, briefly discusses what has been added since the last stable version (Weka 3.4) released in 2003.
19,603 citations
"Aggression recognition using overla..." refers methods in this paper
...Classification for aggression level recognition and PredictedOverlap is performed using a Random Forest classifier with 100 trees as implemented in Weka [13]....
[...]
...In search for the best feature set we applied the feature subset evaluation of Weka [13]....
[...]
TL;DR: In this article, a method of over-sampling the minority class involves creating synthetic minority class examples, which is evaluated using the area under the Receiver Operating Characteristic curve (AUC) and the ROC convex hull strategy.
Abstract: An approach to the construction of classifiers from imbalanced datasets is described. A dataset is imbalanced if the classification categories are not approximately equally represented. Often real-world data sets are predominately composed of "normal" examples with only a small percentage of "abnormal" or "interesting" examples. It is also the case that the cost of misclassifying an abnormal (interesting) example as a normal example is often much higher than the cost of the reverse error. Under-sampling of the majority (normal) class has been proposed as a good means of increasing the sensitivity of a classifier to the minority class. This paper shows that a combination of our method of oversampling the minority (abnormal)cla ss and under-sampling the majority (normal) class can achieve better classifier performance (in ROC space)tha n only under-sampling the majority class. This paper also shows that a combination of our method of over-sampling the minority class and under-sampling the majority class can achieve better classifier performance (in ROC space)t han varying the loss ratios in Ripper or class priors in Naive Bayes. Our method of over-sampling the minority class involves creating synthetic minority class examples. Experiments are performed using C4.5, Ripper and a Naive Bayes classifier. The method is evaluated using the area under the Receiver Operating Characteristic curve (AUC)and the ROC convex hull strategy.
17,313 citations
TL;DR: In this article, a method of over-sampling the minority class involves creating synthetic minority class examples, which is evaluated using the area under the Receiver Operating Characteristic curve (AUC) and the ROC convex hull strategy.
Abstract: An approach to the construction of classifiers from imbalanced datasets is described. A dataset is imbalanced if the classification categories are not approximately equally represented. Often real-world data sets are predominately composed of "normal" examples with only a small percentage of "abnormal" or "interesting" examples. It is also the case that the cost of misclassifying an abnormal (interesting) example as a normal example is often much higher than the cost of the reverse error. Under-sampling of the majority (normal) class has been proposed as a good means of increasing the sensitivity of a classifier to the minority class. This paper shows that a combination of our method of over-sampling the minority (abnormal) class and under-sampling the majority (normal) class can achieve better classifier performance (in ROC space) than only under-sampling the majority class. This paper also shows that a combination of our method of over-sampling the minority class and under-sampling the majority class can achieve better classifier performance (in ROC space) than varying the loss ratios in Ripper or class priors in Naive Bayes. Our method of over-sampling the minority class involves creating synthetic minority class examples. Experiments are performed using C4.5, Ripper and a Naive Bayes classifier. The method is evaluated using the area under the Receiver Operating Characteristic curve (AUC) and the ROC convex hull strategy.
11,512 citations
TL;DR: Turn-taking is used for the ordering of moves in games, for allocating political office, for regulating traffic at intersections, for the servicing of customers at business establishments, and for talking in interviews, meetings, debates, ceremonies, conversations.
Abstract: Publisher Summary Turn taking is used for the ordering of moves in games, for allocating political office, for regulating traffic at intersections, for the servicing of customers at business establishments, and for talking in interviews, meetings, debates, ceremonies, conversations. This chapter discusses the turn-taking system for conversation. On the basis of research using audio recordings of naturally occurring conversations, the chapter highlights the organization of turn taking for conversation and extracts some of the interest that organization has. The turn-taking system for conversation can be described in terms of two components and a set of rules. These two components are turn-constructional component and turn-constructional component. Turn-allocational techniques are distributed into two groups: (1) those in which next turn is allocated by current speaker selecting a next speaker and (2) those in which next turn is allocated by self-selection. The turn-taking rule-set provides for the localization of gap and overlap possibilities at transition-relevance places and their immediate environment, cleansing the rest of a turn's space of systematic bases for their possibility.
10,944 citations
01 Jan 2002
8,837 citations
"Aggression recognition using overla..." refers methods in this paper
...The software tool Praat [2] was used to extract these features....
[...]