“Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告

Breakthroughs from the field of deep learning are radically changing how sensor data are interpreted to extract the high-level information needed by mobile apps. It is critical that the gains in inference accuracy that deep models afford become embedded in future generations of mobile apps. In this work, we present the design and implementation of DeepX, a software accelerator for deep learning execution. DeepX signif- icantly lowers the device resources (viz. memory, computation, energy) required by deep learning that currently act as a severe bottleneck to mobile adoption. The foundation of DeepX is a pair of resource control algorithms, designed for the inference stage of deep learning, that: (1) decompose monolithic deep model network architectures into unit- blocks of various types, that are then more efficiently executed by heterogeneous local device processors (e.g., GPUs, CPUs); and (2), perform principled resource scaling that adjusts the architecture of deep models to shape the overhead each unit-blocks introduces. Experiments show, DeepX can allow even large-scale deep learning models to execute efficently on modern mobile processors and significantly outperform existing solutions, such as cloud-based offloading.

/pdf/deepx-a-software-accelerator-for-low-power-deep-learning-w8z2ckxjxt.pdf

DeepX: a software accelerator for low-power deep learning inference on mobile devices

For physical activity recognition, smartphone sensors, such as an accelerometer and a gyroscope, are being utilized in many research studies. So far, particularly, the accelerometer has been extensively studied. In a few recent studies, a combination of a gyroscope, a magnetometer (in a supporting role) and an accelerometer (in a lead role) has been used with the aim to improve the recognition performance. How and when are various motion sensors, which are available on a smartphone, best used for better recognition performance, either individually or in combination? This is yet to be explored. In order to investigate this question, in this paper, we explore how these various motion sensors behave in different situations in the activity recognition process. For this purpose, we designed a data collection experiment where ten participants performed seven different activities carrying smart phones at different positions. Based on the analysis of this data set, we show that these sensors, except the magnetometer, are each capable of taking the lead roles individually, depending on the type of activity being recognized, the body position, the used data features and the classification method employed (personalized or generalized). We also show that their combination only improves the overall recognition performance when their individual performances are not very high, so that there is room for performance improvement. We have made our data set and our data collection application publicly available, thereby making our experiments reproducible.

/pdf/fusion-of-smartphone-motion-sensors-for-physical-activity-4xdnlyv3ro.pdf

Fusion of smartphone motion sensors for physical activity recognition.

Microphones are remarkably powerful sensors of human behavior and context. However, audio sensing is highly susceptible to wild fluctuations in accuracy when used in diverse acoustic environments (such as, bedrooms, vehicles, or cafes), that users encounter on a daily basis. Towards addressing this challenge, we turn to the field of deep learning; an area of machine learning that has radically changed related audio modeling domains like speech recognition. In this paper, we present DeepEar -- the first mobile audio sensing framework built from coupled Deep Neural Networks (DNNs) that simultaneously perform common audio sensing tasks. We train DeepEar with a large-scale dataset including unlabeled data from 168 place visits. The resulting learned model, involving 2.3M parameters, enables DeepEar to significantly increase inference robustness to background noise beyond conventional approaches present in mobile devices. Finally, we show DeepEar is feasible for smartphones by building a cloud-free DSP-based prototype that runs continuously, using only 6% of the smartphone's battery daily.

/pdf/deepear-robust-smartphone-audio-sensing-in-unconstrained-1w69h7mmer.pdf

DeepEar: robust smartphone audio sensing in unconstrained acoustic environments using deep learning

The effect of consumer innovativeness on perceived value and continuance intention to use smartwatch

Mobile applications that sense continuously, such as location monitoring, are emerging. Despite their usefulness, their adoption in real-world deployment situations has been extremely slow. Many smartphone users are turned away by the drastic battery drain caused by continuous sensing and processing. Also, the extractable contexts from the phone are quite limited due to its position and sensing modalities. In this paper, we propose CoMon, a novel cooperative ambience monitoring platform, which newly addresses the energy problem through opportunistic cooperation among nearby mobile users. To maximize the benefit of cooperation, we develop two key techniques, (1) continuity-aware cooperator detection and (2) benefit-aware negotiation. The former employs heuristics to detect cooperators who will remain in the vicinity for a long period of time, while the latter automatically devises a cooperation plan that provides mutual benefit to cooperators, while considering running applications, available devices, and user policies. Through continuity- and benefit-aware operation, CoMon enables applications to monitor the environment at much lower energy consumption. We implement and deploy a CoMon prototype and show that it provides significant benefit for mobile sensing applications.

https://ink.library.smu.edu.sg/cgi/viewcontent.cgi?article=3066&context=sis_research

CoMon: cooperative ambience monitoring platform with continuity and benefit awareness

In this paper, we propose SocioPhone, a novel initiative to build a mobile platform for face-to-face interaction monitoring. Face-to-face interaction, especially conversation, is a fundamental part of everyday life. Interaction-aware applications aimed at facilitating group conversations have been proposed, but have not proliferated yet. Useful contexts to capture and support face-to-face interactions need to be explored more deeply. More important, recognizing delicate conversational contexts with commodity mobile devices requires solving a number of technical challenges. As a first step to address such challenges, we identify useful meta-linguistic contexts of conversation, such as turn-takings, prosodic features, a dominant participant, and pace. These serve as cornerstones for building a variety of interaction-aware applications. SocioPhone abstracts such useful meta-linguistic contexts as a set of intuitive APIs. Its runtime efficiently monitors registered contexts during in-progress conversations and notifies applications on-the-fly. Importantly, we have noticed that online turn monitoring is the basic building block for extracting diverse meta-linguistic contexts, and have devised a novel volume-topography-based method. We show the usefulness of SocioPhone with several interesting applications: SocioTherapist, SocioDigest, and Tug-of-War. Also, we show that our turn-monitoring technique is highly accurate and energy-efficient under diverse real-life situations.

https://ink.library.smu.edu.sg/cgi/viewcontent.cgi?article=3063&context=sis_research

SocioPhone: everyday face-to-face interaction monitoring platform using multi-phone sensor fusion

The rise of consumer wearables promises to have a profound impact on peoples lives by going beyond counting steps. Wearables such as eSense-an in-ear multisensory stereo device-for personal-scale behavior analytics could help accelerate our understanding of a wide range of human activities in a nonintrusive manner.

Earables for Personal-Scale Behavior Analytics

In this paper, we present Orchestrator, an active resource orchestration framework for mobile context monitoring Emerging pervasive environments will introduce a PAN-scale sensor-rich mobile platform consisting of a mobile device and many wearable and space-embedded sensors In such environments, it is challenging to enable multiple context-aware applications requiring continuous context monitoring to simultaneously run and share highly scarce and dynamic resources Orchestrator enables multiple applications to effectively share the resources while exploiting the full capacity of overall system resources and providing high-quality service to users For effective orchestration, we propose an active resource use orchestration approach that actively finds appropriate resource uses for applications and flexibly utilizes them depending on dynamic system conditions Orchestrator is built upon a prototype platform that consists of off-the-shelf mobile devices and sensor motes We present the detailed design, implementation, and evaluation of Orchestrator The evaluation results show that Orchestrator enables applications in a resource-efficient way

Orchestrator: An active resource orchestration framework for mobile context monitoring in sensor-rich mobile environments

User context is defined by data generated through everyday physical activity in sensor-rich, resource-limited mobile environments.

https://ink.library.smu.edu.sg/cgi/viewcontent.cgi?article=3067&context=sis_research

Chulhong Min

Papers

CoMon: cooperative ambience monitoring platform with continuity and benefit awareness

SocioPhone: everyday face-to-face interaction monitoring platform using multi-phone sensor fusion

Earables for Personal-Scale Behavior Analytics

Orchestrator: An active resource orchestration framework for mobile context monitoring in sensor-rich mobile environments

MobiCon: a mobile context-monitoring platform