Showing papers by "Sebastian Möller published in 2023"

Real-time affect detection in virtual reality: a technique based on a three-dimensional model of affect and EEG signals

Abstract: This manuscript explores the development of a technique for detecting the affective states of Virtual Reality (VR) users in real-time. The technique was tested with data from an experiment where 18 participants observed 16 videos with emotional content inside a VR home theater, while their electroencephalography (EEG) signals were recorded. Participants evaluated their affective response toward the videos in terms of a three-dimensional model of affect. Two variants of the technique were analyzed. The difference between both variants was the method used for feature selection. In the first variant, features extracted from the EEG signals were selected using Linear Mixed-Effects (LME) models. In the second variant, features were selected using Recursive Feature Elimination with Cross Validation (RFECV). Random forest was used in both variants to build the classification models. Accuracy, precision, recall and F1 scores were obtained by cross-validation. An ANOVA was conducted to compare the accuracy of the models built in each variant. The results indicate that the feature selection method does not have a significant effect on the accuracy of the classification models. Therefore, both variations (LME and RFECV) seem equally reliable for detecting affective states of VR users. The mean accuracy of the classification models was between 87% and 93%.

Effects of user factors on user experience in virtual reality: age, gender, and VR experience as influencing factors for VR exergames

Abstract: Abstract As virtual reality (VR) technology is extensively developing in past years, more and more people are using it in different fields. One of the fast-developing areas in VR is exergaming, a combination of physical exercise and a game. VR exergames that aim to engage people in physical activity should look and feel good for users regardless of their age, gender, or their previous VR experience with similar technologies. However, recent studies showed that those factors are influencing the user experience (UX) with virtual reality. Building on top of the initial study that has reported on the effect of human influencing factors for exergaming, with this work, we investigated the influence of user parameters (such as age, gender, and previous VR experience) on their motivation for sports and VR exergaming. The study was done using a crowdsourcing platform to recruit a diverse set of participants, with the aim to explore how different user factors are connected to sports motivation. Results show significant differences in the user’s sports motivation and affinity for technology interaction depending on the age group, gender, previous experience with VR, their weekly exercise routine, and how much money they spend on sports yearly.

Comparing Simulated and Real Conversations for QoE Assessments: Insights from ARKit-Based Facial Configuration Analyses

Abstract: This manuscript investigates the suitability of video-based conversation simulations for studying human reactions to quality degradations, focusing on facial configurations as a proxy for QoE. We analyze data from two distinct studies: a video-simulated video-telephony scenario using the storytime dataset, where participants passively watched videos, and a second study involving real conversations between participants. In both studies, facial features were continuously recorded using Apple's iOS ARKit API. We identify a factor structure of facial features that significantly relates to participants' QoE ratings in the first study and validate its robustness by replicating it in the second, independent study. Our findings suggest statistically significant estimations of QoE ratings across both paradigms, demonstrating the suitability of passive conversation simulations for studying human reactions to quality degradation. We assess the value of the proposed approach at its present stage and conclude that it can be a valuable tool when used in conjunction with other methods, as its predictive capabilities are still not robust enough to rely solely on this analysis technique.

Exploring users' sense of safety in public using an Augmented Reality application

Abstract: Nowadays, Augmented Reality (AR) is available on almost all smartphones creating some exciting interaction opportunities but also challenges. For example, already after the famous AR app Pokemon GO was released in July 2016, numerous accidents related to the use of the app were reported by users. At the same time, the spread of AR can be noticed in the tourism industry, enabling tourists to explore their surroundings in new ways but also exposing them to safety issues. This preliminary study explores users' sense of safety when manipulating the amount and UI elements visualization parameters of Point of Interest (POI) markers in a developed AR application. The results show that the amount of POI markers that are displayed is significant for participants' sense of safety. The influence of manipulating UI elements in terms of “transparency”, “color”, and “size” cannot be proven. Nevertheless, most tested people stated that manipulating transparency and size somehow influences their sense of safety, so a closer look at them should be taken in future studies.

Exploring the Mobile Usability of Argumentative Dialogue Systems for Opinion Building

Abstract: Nowadays speech-driven interfaces such as mobile digital assistants and chatbots can support collaborative information seeking and are becoming increasingly commonplace. Especially, mobile dialogue assistants offer innovative approaches to deliver and access information and thus, display a promising approach to assist humans in their opinion building process. Still, due to the complexity of argumentative tasks mobile argumentative speech interfaces are still very scarce. Hence, the effect of such interfaces on a user’s opinion building process is quite unexplored. In this paper, we investigate the influence of such interfaces on the interest and opinion building process of users. Both categories Therefore we introduce two (I/O) modalities (menu/speech) of the argumentative dialog system (ADS) BEA (“Building Engaging Argumentation” [2]) which enables the user to scrutinize arguments on both sides of a controversial topic. In particular, we reflect on the influence and advantages of a spoken hands-free versus a clickable drop-down menu-based ADS with regard to “mobile” dialog systems use cases. Therefore the users’ expectations and experiences in a self-assessment questionnaire are evaluated and discussed in comparison to our user interest and opinion model.

MultiTACRED: A Multilingual Version of the TAC Relation Extraction Dataset

Abstract: Relation extraction (RE) is a fundamental task in information extraction, whose extension to multilingual settings has been hindered by the lack of supervised resources comparable in size to large English datasets such as TACRED (Zhang et al., 2017). To address this gap, we introduce the MultiTACRED dataset, covering 12 typologically diverse languages from 9 language families, which is created by machine-translating TACRED instances and automatically projecting their entity annotations. We analyze translation and annotation projection quality, identify error categories, and experimentally evaluate fine-tuned pretrained mono- and multilingual language models in common transfer learning scenarios. Our analyses show that machine translation is a viable strategy to transfer RE instances, with native speakers judging more than 83% of the translated instances to be linguistically and semantically acceptable. We find monolingual RE model performance to be comparable to the English original for many of the target languages, and that multilingual models trained on a combination of English and target language data can outperform their monolingual counterparts. However, we also observe a variety of translation and annotation projection errors, both due to the MT systems and linguistic features of the target languages, such as pronoun-dropping, compounding and inflection, that degrade dataset quality and RE model performance.

Corrigendum: Real-time affect detection in virtual reality: A technique based on a three-dimensional model of affect and EEG signals

Abstract: Quality and Usability Lab, Institute for Software Technology and Theoretical Computer Science, Faculty of Electrical Engineering and Computer Science, Technische Universitat Berlin, Berlin, Germany, UTS Games Studio, Faculty of Engineering and IT, University of Technology Sydney UTS, Sydney, NSW, Australia, German Research Center for Artificial Intelligence (DFKI), Berlin, Germany, Hamm-Lippstadt University of Applied Sciences, Hamm, Germany

The Future of the Performing Arts with Extended Reality

Abstract: Extended Reality (XR), an umbrella term for AR and VR, is growing in academic and business applications. Each technology simulates reality differently regarding immersion, content, and environment interaction. This gives content providers various chances to deliver material in innovative ways and from faraway locations. XR’s freedom allows for a lot of creativity in different aspects, including such as design and art. XR technology has already been included and used in several events and venues recently, but this work will discuss possibilities with XR and the performing arts. During the last several years, XR and performing arts have gotten closer. The power and necessity to integrate developing technologies into arts are becoming more apparent, especially with COVID-19. XR technology’s expansion and the performing arts digital transition provide a dilemma: how to construct the future of performing arts with XR? This study theoretically addresses the challenge by building the prototype experience and testing this prototype XR performance with a live audience.

Klinische Entscheidungsfindung mit Künstlicher Intelligenz

Abstract: Dieses Open-Access-Buch bietet einen verständlichen Überblick über Entscheidungsunterstützungssysteme in der klinischen Praxis.

Embedding Self-Assessment Manikins in Mobile Augmented Reality Applications

Abstract: Augmented Reality (AR) applications have been widely investigated over the past decades, however, due to the simultaneous advancements in technology, the research has to be constantly evolving, too. Specifically, if looking at aspects that influence User Experience (UX) and Usability, newer technologies in AR-hardware, as well as in evaluation methods, offer promising features that could have a positive influence but might also uncover new challenges. The following paper is going to propose a prototype for the assessment of emotional responses to provided stimuli with Self-Assessment Manikin (SAM) to be integrated within an AR application. For that, the first important aspects of AR will be introduced, defined, and at times complemented and compared with known concepts from Virtual Reality. After assessing the current state-of-the-art by investigating related literature, the motivation behind an integrated approach will be explained in more detail and the implementation of the SAM prototype with the UI design platform Figma will be introduced. Results from the conducted user study will provide valuable insight into how the three-dimensional SAM was received and consider the opportunities and limitations of the implemented design.

Fighting Disinformation: Overview of Recent AI-Based Collaborative Human-Computer Interaction for Intelligent Decision Support Systems

Abstract: : Methods for automatic disinformation detection have gained much attention in recent years, as false information can have a severe impact on societal cohesion. Disinformation can influence the outcome of elections, the spread of diseases by preventing adequate countermeasures adoption, and the formation of allies, as the Russian invasion in Ukraine has shown. Hereby, not only text as a medium but also audio recordings, video content, and images need to be taken into consideration to fight fake news. However, automatic fact-checking tools cannot handle all modalities at once and face difficulties embedding the context of information, sarcasm, irony, and when there is no clear truth value. Recent research has shown that collaborative human-machine systems can identify false information more successfully than human or machine learning methods alone. Thus, in this paper, we present a short yet comprehensive state of current automatic disinformation detection approaches for text, audio, video, images, multimodal combinations, their extension into intelligent decision support systems (IDSS) as well as forms and roles of human collaborative co-work. In real life, such systems are increasingly applied by journalists, setting the specifications to human roles according to two most prominent types of use cases, namely daily news dossiers and investigative journalism .

What is Your Location Privacy Worth? Monetary Valuation of Different Location Types and Privacy Influencing Factors

Abstract: Nowadays, many apps use location data to estimate the user's behavior for targeted advertising, predicting significant locations, personal preferences, state of health, and sports activities. Users of location-based services are often left with no other choice than to accept or reject location tracking when they want to use various applications. Especially, users with higher privacy concerns may reduce the frequency of location tracking by turning it off in the settings. However, most users are unaware that many applications installed on their phones are continuously tracking them. Therefore, this study attempts to answer how (obviously) being tracked over one-week influences a user's privacy concerns. The study was implemented using an iOS app, which participants could install on their smartphones. Moreover, over one week, the participants were requested to answer daily mini-questionnaires about how much they would be willing to pay for the protection of their location information on a monthly basis and how much money they were willing to accept in exchange for their location information. Hereby, the context was an important criterion to determine how the monetary values vary among different location types for, among others, home location, work location, and meeting family and friends. The participants (N=51) interacted with the app on a daily basis by filling out various daily mini-surveys based on their significant locations visited. The results show a significant difference between the monetary valuating of willingness to pay and to accept for all location types except work location and sharing scenarios contributing to further empirical evidence for the endowment effect. The obvious fact of continuously being tracked did not increase the privacy concern of participants.

Evaluating a Spoken Argumentative Dialogue System

Abstract: A natural way for humans to build an opinion on a topic is through the gathering and exchange of new arguments. Speech interfaces for argumentative dialogue systems (ADS) are rather scarce and quite complex. To provide a more natural and intuitive interface, we include an adaption of a recently introduced natural language understanding (NLU) framework tailored to argumentative tasks into a complete end-to-end ADS. Within this paper we investigate the influence of two different input/output modalities (speech/speech and drop-down menu/text) and discuss issues and problems we encountered in a user study with 202 participants using our ADS.

Unraveling the Hangry Rater: Non-linear Effects of Hunger on Multimedia Quality Perception

Abstract: The subjective quality of experience (QoE) in multimedia contexts is influenced by various factors, including individual differences among raters and experimental setups. While the latter has been extensively studied, the former remains relatively unexplored. This paper investigates the impact of hangriness - a mental state of irritability and frustration caused by hunger - on QoE ratings. In our analysis, hangriness appears to be prevalent in a specific time interval, where individuals have not consumed any food between five and eleven hours. Our analysis, comprising ratings from 100 participants, reveals a significant, non-linear effect of hangriness on QoE ratings, specifically for multimedia stimuli with subpar quality. Participants in the hangry state rated such stimuli significantly worse compared to those who had eaten recently or abstained from food for more than eleven hours. Interestingly, this effect was not observed for high-quality multimedia content. Our findings highlight the importance of considering individual differences, such as hangriness, in QoE research, as they can significantly impact subjective ratings. Further research is needed to corroborate these results and explore other factors that may influence QoE ratings. This work contributes to a better understanding of individual variability in multimedia quality perception and provides insights for designing more reliable QoE assessment methods.

An Investigation of the Influence of Ambient Noise on User Experience in Virtual Reality

Abstract: Although there are several aspects that might impact one's feeling of immersion or presence in a virtual environment, noises in the actual world are often overlooked. Distractions (such as noises or lights) may have an effect on a person's mental or emotional state in real life, and these interruptions can be disturbing while trying to focus on work. With a steady stream of distractions, one may eventually lose interest and make mistakes or stop the activity. The goal of this research is to better understand the impact of various ambient sounds on presence, immersion, and decision-making by using virtual reality and audio rendering technology. This study's sound sources were obtained and recorded from several real-life sources before being mixed into the virtual world. These audio recordings have been shown to influence a person's decision-making skills by generating a sudden shift in different emotional states (such as annoyance and anger). In certain circumstances, binaural recording is used to provide the listener with a 3D stereo sound experience. They were triggered at random intervals while the user was in the virtual environment. Based on the findings of the study presented in this paper, some types of noises, such as continuous and impulsive noise, have a negative impact on user experience by reducing immersion in the virtual reality environment. It was also discovered that the more interaction there was, the more mental work was required to complete the activities.

Protect and Extend - Using GANs for Synthetic Data Generation of Time-Series Medical Records

Abstract: Preservation of private user data is of paramount importance for high Quality of Experience (QoE) and acceptability, particularly with services treating sensitive data, such as IT-based health services. Whereas anonymization techniques were shown to be prone to data re-identification, synthetic data generation has gradually replaced anonymization since it is relatively less time and resource-consuming and more robust to data leakage. Generative Adversarial Networks (GANs) have been used for generating synthetic datasets, especially GAN frameworks adhering to the differential privacy phenomena. This research compares state-of-the-art GAN-based models for synthetic data generation to generate time-series synthetic medical records of dementia patients which can be distributed without privacy concerns. Predictive modeling, autocorrelation, and distribution analysis are used to assess the Quality of Generating (QoG) of the generated data. The privacy preservation of the respective models is assessed by applying membership inference attacks to determine potential data leakage risks. Our experiments indicate the superiority of the privacy-preserving GAN (PPGAN) model over other models regarding privacy preservation while maintaining an acceptable level of QoG. The presented results can support better data protection for medical use cases in the future.