Consumer Wearables and Affective Computing for Wellbeing Support
07 Dec 2020-pp 482-487
TL;DR: In this article, the WellAff system was proposed to recognize affective states for wellbeing support in patients suffering from bipolar disorder, in particular patients with chronic kidney disease and bipolar disorder.
Abstract: Wearables equipped with pervasive sensors enable us to monitor physiological and behavioral signals in our everyday life. We propose the WellAff system able to recognize affective states for wellbeing support. It also includes health care scenarios, in particular patients with chronic kidney disease suffering from bipolar disorders. For the need of a large-scale field study, we revised over 50 off-the-shelf devices in terms of usefulness for emotion, stress, meditation, sleep, and physical activity recognition and analysis. Their usability directly comes from the types of sensors they possess as well as the quality and availability of raw signals. We found there is no versatile device suitable for all purposes. Using Empatica E4 and Samsung Galaxy Watch, we have recorded physiological signals from 11 participants over many weeks. The gathered data enabled us to train a classifier that accurately recognizes strong affective states.
Citations
More filters
TL;DR: In this paper , the Emognition dataset is dedicated to testing methods for emotion recognition (ER) from physiological responses and facial expressions from short film clips eliciting nine discrete emotions: amusement, awe, enthusiasm, liking, surprise, anger, disgust, fear, and sadness.
Abstract: The Emognition dataset is dedicated to testing methods for emotion recognition (ER) from physiological responses and facial expressions. We collected data from 43 participants who watched short film clips eliciting nine discrete emotions: amusement, awe, enthusiasm, liking, surprise, anger, disgust, fear, and sadness. Three wearables were used to record physiological data: EEG, BVP (2x), HR, EDA, SKT, ACC (3x), and GYRO (2x); in parallel with the upper-body videos. After each film clip, participants completed two types of self-reports: (1) related to nine discrete emotions and (2) three affective dimensions: valence, arousal, and motivation. The obtained data facilitates various ER approaches, e.g., multimodal ER, EEG- vs. cardiovascular-based ER, discrete to dimensional representation transitions. The technical validation indicated that watching film clips elicited the targeted emotions. It also supported signals' high quality.
13 citations
TL;DR: In this paper , the Emognition dataset is dedicated to testing methods for emotion recognition (ER) from physiological responses and facial expressions from short film clips eliciting nine discrete emotions: amusement, awe, enthusiasm, liking, surprise, anger, disgust, fear, and sadness.
Abstract: The Emognition dataset is dedicated to testing methods for emotion recognition (ER) from physiological responses and facial expressions. We collected data from 43 participants who watched short film clips eliciting nine discrete emotions: amusement, awe, enthusiasm, liking, surprise, anger, disgust, fear, and sadness. Three wearables were used to record physiological data: EEG, BVP (2x), HR, EDA, SKT, ACC (3x), and GYRO (2x); in parallel with the upper-body videos. After each film clip, participants completed two types of self-reports: (1) related to nine discrete emotions and (2) three affective dimensions: valence, arousal, and motivation. The obtained data facilitates various ER approaches, e.g., multimodal ER, EEG- vs. cardiovascular-based ER, discrete to dimensional representation transitions. The technical validation indicated that watching film clips elicited the targeted emotions. It also supported signals' high quality.
8 citations
22 Mar 2021
TL;DR: In this article, a pre-trained binary model recognizing physiologically arousing events in real-time and triggering self-assessments at a convenient point in time is proposed.
Abstract: Emotion and affect recognition in the uncontrolled everyday-life environment remains challenging. One of its vital problems is collecting numerous annotated emotional samples indispensable for learning the reasoning model. We propose a novel method supporting rich emotional data collection - a pre-trained binary model recognizing physiologically arousing events in real-time and triggering self-assessments at a convenient point in time. An experimental study on 6.000 hours of recorded physiological signals has been performed. The results suggest that we are able to detect emotional events in real-life scenarios to enhance data collection for emotion recognition in the field.
3 citations
TL;DR: In this article , the authors present a guide for the incomers in the field on how to design digital health interventions with case studies from the Cancer Better Life Experience (CAPABLE) European project.
3 citations
21 Mar 2022
TL;DR: In this article , the cold start problem was explored for emotion recognition in real life from physiological signals provided by wrist worn devices, where no data from the target subjects (users) were available at the beginning of the experiment to train the reasoning model.
Abstract: Emotion recognition in real life from physiological signals provided by wrist worn devices still remains a great challenge especially due to difficulties with gathering annotated emotional events. For that purpose, we suggest building pre-trained machine learning models capable of detecting intense emotional states. This work aims to explore the cold start problem, where no data from the target subjects (users) are available at the beginning of the experiment to train the reasoning model. To address this issue, we investigate the potential of per-group personalization and the amount of data needed to perform it. Our results on real-life data indicate that even a week’s worth of personalized data improves the model performance.
2 citations
References
More filters
TL;DR: Ecological momentary assessment holds unique promise to advance the science and practice of clinical psychology by shedding light on the dynamics of behavior in real-world settings.
Abstract: Assessment in clinical psychology typically relies on global retrospective self-reports collected at research or clinic visits, which are limited by recall bias and are not well suited to address how behavior changes over time and across contexts. Ecological momentary assessment (EMA) involves repeated sampling of subjects’ current behaviors and experiences in real time, in subjects’ natural environments. EMA aims to minimize recall bias, maximize ecological validity, and allow study of microprocesses that influence behavior in real-world contexts. EMA studies assess particular events in subjects’ lives or assess subjects at periodic intervals, often by random time sampling, using technologies ranging from written diaries and telephones to electronic diaries and physiological sensors. We discuss the rationale for EMA, EMA designs, methodological and practical issues, and comparisons of EMA and recall data. EMA holds unique promise to advance the science and practice of clinical psychology by shedding ligh...
4,286 citations
TL;DR: This paper explores how these various motion sensors behave in different situations in the activity recognition process, and shows that they are each capable of taking the lead roles individually, depending on the type of activity being recognized, the body position, the used data features and the classification method employed.
Abstract: For physical activity recognition, smartphone sensors, such as an accelerometer and a gyroscope, are being utilized in many research studies. So far, particularly, the accelerometer has been extensively studied. In a few recent studies, a combination of a gyroscope, a magnetometer (in a supporting role) and an accelerometer (in a lead role) has been used with the aim to improve the recognition performance. How and when are various motion sensors, which are available on a smartphone, best used for better recognition performance, either individually or in combination? This is yet to be explored. In order to investigate this question, in this paper, we explore how these various motion sensors behave in different situations in the activity recognition process. For this purpose, we designed a data collection experiment where ten participants performed seven different activities carrying smart phones at different positions. Based on the analysis of this data set, we show that these sensors, except the magnetometer, are each capable of taking the lead roles individually, depending on the type of activity being recognized, the body position, the used data features and the classification method employed (personalized or generalized). We also show that their combination only improves the overall recognition performance when their individual performances are not very high, so that there is room for performance improvement. We have made our data set and our data collection application publicly available, thereby making our experiments reproducible.
426 citations
TL;DR: This review has summarized the features and evaluated the characteristics of a cross-section of technologies for health and sports performance according to what the technology is claimed to do, whether it has been validated and is reliable, and if it is suitable for general consumer use.
Abstract: The commercial market for technologies to monitor and improve personal health and sports performance is ever expanding. A wide range of smart watches, bands, garments, and patches with embedded sensors, small portable devices and mobile applications now exist to record and provide users with feedback on many different physical performance variables. These variables include cardiorespiratory function, movement patterns, sweat analysis, tissue oxygenation, sleep, emotional state, and changes in cognitive function following concussion. In this review, we have summarized the features and evaluated the characteristics of a cross-section of technologies for health and sports performance according to what the technology is claimed to do, whether it has been validated and is reliable, and if it is suitable for general consumer use. Consumers who are choosing new technology should consider whether it (1) produces desirable (or non-desirable) outcomes, (2) has been developed based on real-world need, and (3) has been tested and proven effective in applied studies in different settings. Among the technologies included in this review, more than half have not been validated through independent research. Only 5% of the technologies have been formally validated. Around 10% of technologies have been developed for and used in research. The value of such technologies for consumer use is debatable, however, because they may require extra time to set up and interpret the data they produce. Looking to the future, the rapidly expanding market of health and sports performance technology has much to offer consumers. To create a competitive advantage, companies producing health and performance technologies should consult with consumers to identify real-world need, and invest in research to prove the effectiveness of their products. To get the best value, consumers should carefully select such products, not only based on their personal needs, but also according to the strength of supporting evidence and effectiveness of the products.
307 citations
TL;DR: This survey will examine the recent works on stress detection in daily life which are using smartphones and wearable devices and investigate the works according to used physiological modality and their targeted environment such as office, campus, car and unrestricted daily life conditions.
255 citations
TL;DR: The impact of stress to multiple bodily responses is surveyed and efficiency, robustness and consistency of biosignal data features across the current state of knowledge in stress detection are put on.
Abstract: This review investigates the effects of psychological stress on the human body measured through biosignals. When a potentially threatening stimulus is perceived, a cascade of physiological processes occurs mobilizing the body and nervous system to confront the imminent threat and ensure effective adaptation. Biosignals that can be measured reliably in relation to such stressors include physiological (EEG, ECG, EDA, EMG) and physical measures (respiratory rate, speech, skin temperature, pupil size, eye activity). A fundamental objective in this area of psychophysiological research is to establish reliable biosignal indices that reveal the underlying physiological mechanisms of the stress response. Motivated by the lack of comprehensive guidelines on the relationship between the multitude of biosignal features used in the literature and their corresponding behaviour during stress, in this paper, the impact of stress to multiple bodily responses is surveyed. Emphasis is put on the efficiency, robustness and consistency of biosignal data features across the current state of knowledge in stress detection. It is also explored multimodal biosignal analysis and modelling methods for deriving accurate stress correlates. This paper aims to provide a comprehensive review on biosignal patterns caused during stress conditions and reliable practical guidelines towards more efficient detection of stress.
243 citations