The performance of the Mel-Frequency Cepstrum Coefficients (MFCC) may be affected by (1) the number of filters, (2) the shape of filters, (3) the way in which filters are spaced, and (4) the way in which the power spectrum is warped. In this paper, several comparison experiments are done to find a best implementation. The traditional MFCC calculation excludes the 0th coefficient for the reason that it is regarded as somewhat unreliable. According to the analysis and experiments, the authors find that it can be regarded as the generalized frequency band energy (FBE) and is hence useful, which results in the FBE-MFCC. The authors also propose a better analysis, namely the auto-regressive analysis, on the frame energy,which outperforms its 1st and/or 2nd order differential derivatives. Experiments with the “863”Speech Database show that, compared with the traditional MFCC with its corresponding autoregressive analysis coefficients, the FBE-MFCC and the frame energy with their corresponding auto-regressive analysis coefficients form the best combination, reducing the Chinese syllable error rate (CSER) by about 10%, while the FBE-MFCC with the corresponding auto-regressive analysis coefficients reduces CSER by 2.5%. Comparison experiments are also done with a quite casual Chinese speech database, named Chinese Annotated Spontaneous Speech (CASS)corpus. The FBE-MFCC can reduce the error rate by about 2.9% on an average.

Comparison of Different Implementations of MFCC

Theory building with big data-driven research – Moving away from the “What” towards the “Why”

Performance Enhancement of Flexible Piezoelectric Nanogenerator via Doping and Rational 3D Structure Design For Self‐Powered Mechanosensational System

Recognizing emotions during social interactions has many potential applications with the popularization of low-cost mobile sensors, but a challenge remains with the lack of naturalistic affective interaction data. Most existing emotion datasets do not support studying idiosyncratic emotions arising in the wild as they were collected in constrained environments. Therefore, studying emotions in the context of social interactions requires a novel dataset, and K-EmoCon is such a multimodal dataset with comprehensive annotations of continuous emotions during naturalistic conversations. The dataset contains multimodal measurements, including audiovisual recordings, EEG, and peripheral physiological signals, acquired with off-the-shelf devices from 16 sessions of approximately 10-minute long paired debates on a social issue. Distinct from previous datasets, it includes emotion annotations from all three available perspectives: self, debate partner, and external observers. Raters annotated emotional displays at intervals of every 5 seconds while viewing the debate footage, in terms of arousal-valence and 18 additional categorical emotions. The resulting K-EmoCon is the first publicly available emotion dataset accommodating the multiperspective assessment of emotions during social interactions.

/pdf/k-emocon-a-multimodal-sensor-dataset-for-continuous-emotion-1mqz29rjdh.pdf

K-EmoCon, a multimodal sensor dataset for continuous emotion recognition in naturalistic conversations

/pdf/k-emocon-a-multimodal-sensor-dataset-for-continuous-emotion-4krrqxps2i.pdf

The use of mobile, wearable, and Internet of Things (IoT) technologies fosters unique opportunities for designing novel intelligent positive computing services that address various health and well-being issues such as stress and depression As positive computing research is often cross-disciplinary, it is difficult to acquire holistic perspectives on the design, implementation, and evaluation of intelligent positive computing systems with mobile, wearable, and IoT technologies To bridge this gap, we propose a conceptual framework and review the key components to provide guidelines for intelligent positive computing systems research We also present several practical service scenarios and provide useful insights on opportunities and challenges By critically reflecting on the literature and scenarios, we suggest several research directions on the core topics in intelligent positive computing systems research In addition, we discuss concerns and challenges such as technology dependence, abandonment, side effects, privacy, and ethical issues

Intelligent positive computing with mobile, wearable, and IoT devices: Literature review and research directions

While smartphones have enriched our lives with diverse applications and functionalities, the user experience still often involves manual cumbersome inputs. To purchase a bottle of water for instance, a user must locate an e-commerce app, type the keyword for a search, select the right item from the list, and finally place an order. This process could be greatly simplified if the smartphone identifies the object of interest and automatically executes the user preferred actions for the object. We present Knocker that identifies the object when a user simply knocks on an object with a smartphone. The basic principle of Knocker is leveraging a unique set of responses generated from the knock. Knocker takes a multimodal sensing approach that utilizes microphones, accelerometers, and gyroscopes to capture the knock responses, and exploits machine learning to accurately identify objects. We also present 15 applications enabled by Knocker that showcase the novel interaction method between users and objects. Knocker uses only the built-in smartphone sensors and thus is fully deployable without specialized hardware or tags on either the objects or the smartphone. Our experiments with 23 objects show that Knocker achieves an accuracy of 98% in a controlled lab and 83% in the wild.

Knocker: Vibroacoustic-based Object Recognition with Smartphones

Digital intervention tools against problematic smartphone usage help users control their consumption on smartphones, for example, by setting a time limit on an app. However, today's social media apps offer a mix of quasiessential and addictive features in an app (e.g., Instagram has following feeds, recommended feeds, stories, and direct messaging features), which makes it hard to apply a uniform logic for all uses of an app without a nuanced understanding of feature-level usage behaviors. We study when and why people regret using different features of social media apps on smartphones. We examine regretful feature uses in four smartphone social media apps (Facebook, Instagram, YouTube, and KakaoTalk) by utilizing feature usage logs, ESM surveys on regretful use collected for a week, and retrospective interviews from 29 Android users. In determining whether a feature use is regretful, users considered different types of rewards they obtained from using a certain feature (i.e., social, informational, personal interests, and entertainment) as well as alternative rewards they could have gained had they not used the smartphone (e.g., productivity). Depending on the types of rewards and the way rewards are presented to users, probabilities to regret vary across features of the same app. We highlight three patterns of features with different characteristics that lead to regretful use. First, "following"-based features (e.g., Facebook's News Feed and Instagram's Following Posts and Stories) induce habitual checking and quickly deplete rewards from app use. Second, recommendation-based features situated close to actively used features (e.g., Instagram's Suggested Posts adjacent to Search) cause habitual feature tour and sidetracking from the original intention of app use. Third, recommendation-based features with bite-sized contents (e.g., Facebook's Watch Videos) induce using "just a bit more," making people fall into prolonged use. We discuss implications of our findings for how social media apps and intervention tools can be designed to reduce regretful use and how feature-level usage information can strengthen self-reflection and behavior changes.

/pdf/reflect-not-regret-understanding-regretful-smartphone-use-2jnacoid9r.pdf

Reflect, not Regret: Understanding Regretful Smartphone Use with App Feature-Level Analysis

Federated Learning (FL) enables distributed training of machine learning models while keeping personal data on user devices private. While we witness increasing applications of FL in the area of mobile sensing, such as human activity recognition (HAR), FL has not been studied in the context of a multi-device environment (MDE), wherein each user owns multiple data-producing devices. With the proliferation of mobile and wearable devices, MDEs are increasingly becoming popular in ubicomp settings, therefore necessitating the study of FL in them. FL in MDEs is characterized by being not independent and identically distributed (non-IID) across clients, complicated by the presence of both user and device heterogeneities. Further, ensuring efficient utilization of system resources on FL clients in a MDE remains an important challenge. In this paper, we propose FLAME, a user-centered FL training approach to counter statistical and system heterogeneity in MDEs, and bring consistency in inference performance across devices. FLAME features (i) user-centered FL training utilizing the time alignment across devices from the same user; (ii) accuracy- and efficiency-aware device selection; and (iii) model personalization to devices. We also present an FL evaluation testbed with realistic energy drain and network bandwidth profiles, and a novel class-based data partitioning scheme to extend existing HAR datasets to a federated setup. Our experiment results on three multi-device HAR datasets show that FLAME outperforms various baselines by 4.3-25.8% higher 𝐹 1 score, 1.02-2.86 × greater energy efficiency, and up to 2.06 × speedup in convergence to target accuracy through fair distribution of the FL workload.

/pdf/flame-federated-learning-across-multi-device-environments-2hx1o2j1.pdf

FLAME: Federated Learning Across Multi-device Environments

While mobile instant messaging (MIM) facilitates ubiquitous interpersonal communication, its constant connectivity could build the expectation of an immediate response to messages, and its notifications flood could cause interruptions at inopportune moments. We examine two design concepts for MIM-private status sharing and sender-controlled notifications-that aim to lower the pressure for an immediate reply and reduce unnecessary interruptions by untimely notifications. Private status sharing reactively reveals a customized status with a selected partner(s) only when the partner has sent a message. Sender-controlled notifications give senders the control of choosing whether to send a notification for their own messages. We built MyButler, an Android app prototype that instantiates these two concepts and integrated it with KakaoTalk, a commercial MIM app. During a two-week field study with 11 pairs (5 couples and 6 friend pairs), participants expressed themselves through a total of 210 different statuses, 64.3% of which indicated the current activity or task of the user. Participants reported that private status sharing enabled them to explain their unavailability and relieved the pressure and expectations for timely attendance. We reveal more findings on the types of privately shared statuses and their roles in MIM communication; the in-situ behaviors and patterns of using sender-controlled notifications; and the motivations of MIM users in choosing whether to alert their messages. In terms of message notifications, senders chose to send 25.4% of the messages without any notification. We found that senders' decisions to alert are affected by the receiver's status, their own status to chat, and the possibility of message content exposure to others through notifications. Based on our findings, we draw insights into how the concepts of private status sharing and sender-controlled notifications can be applied in future designs and explorations.

Hyunsung Cho

Papers

Intelligent positive computing with mobile, wearable, and IoT devices: Literature review and research directions

Knocker: Vibroacoustic-based Object Recognition with Smartphones

Reflect, not Regret: Understanding Regretful Smartphone Use with App Feature-Level Analysis

FLAME: Federated Learning Across Multi-device Environments

I Share, You Care: Private Status Sharing and Sender-Controlled Notifications in Mobile Instant Messaging