The widespread presence of motion sensors on users' personal mobile devices has spawned a growing research interest in human activity recognition (HAR). However, when deployed at a large-scale, e.g., on multiple devices, the performance of a HAR system is often significantly lower than in reported research results. This is due to variations in training and test device hardware and their operating system characteristics among others. In this paper, we systematically investigate sensor-, device- and workload-specific heterogeneities using 36 smartphones and smartwatches, consisting of 13 different device models from four manufacturers. Furthermore, we conduct experiments with nine users and investigate popular feature representation and classification techniques in HAR research. Our results indicate that on-device sensor and sensor handling heterogeneities impair HAR performances significantly. Moreover, the impairments vary significantly across devices and depends on the type of recognition technique used. We systematically evaluate the effect of mobile sensing heterogeneities on HAR and propose a novel clustering-based mitigation technique suitable for large-scale deployment of HAR, where heterogeneity of devices and their usage scenarios are intrinsic.

Smart Devices are Different: Assessing and MitigatingMobile Sensing Heterogeneities for Activity Recognition

The ubiquity of communication devices such as smartphones has led to the emergence of context-aware services that are able to respond to specific user activities or contexts. These services allow communication providers to develop new, added-value services for a wide range of applications such as social networking, elderly care and near-emergency early warning systems. At the core of these services is the ability to detect specific physical settings or the context a user is in, using either internal or external sensors. For example, using built-in accelerometers, it is possible to determine whether a user is walking or running at a specific time of day. By correlating this knowledge with GPS data, it is possible to provide specific information services to users with similar daily routines. This article presents a survey of the techniques for extracting this activity information from raw accelerometer data. The techniques that can be implemented in mobile devices range from classical signal processing techniques such as FFT to contemporary string-based methods. We present experimental results to compare and evaluate the accuracy of the various techniques using real data sets collected from daily activities.

/pdf/preprocessing-techniques-for-context-recognition-from-3o0k2lk1mr.pdf

Preprocessing techniques for context recognition from accelerometer data

In this paper, we perform physical motion recognition using mobile phones with built-in accelerometer sensors. Sensor data processing and smoothing techniques are discussed first to reduce the special noise present in phone-collected accelerometer data. We explore orientation-independent features extracted from vertical and horizonal components in acceleration as well as magnitudes of acceleration for six common physical activities, such as sitting, standing, walking, running, driving and bicycling. We find decision tree achieves the best performance among four commonly used static classifiers, while vertical and horizonal features have better recognition accuracy than magnitude features. Furthermore, a well-pruned decision tree with simple time domain features and less over-fitting on the training data can provide a usable model for inferencing a physical activity diary, refined by a similarity match from K-means clustering results and smoothed by an HMM-based Viterbi algorithm.

Toward physical activity diary: motion recognition using simple acceleration features with mobile phones

Obesity is now considered a global epidemic and is predicted to become the number one preventive health threat in the industrialized world. Presently, over 60% of the U.S. adult population is overweight and 30% is obese. This is of concern because obesity is linked to leading causes of death, such as heart and pulmonary diseases, stroke, and type 2 diabetes. The dramatic rise in obesity rates is attributed to an environment that provides easy access to high caloric food and drink and promotes low levels of physical activity. Unfortunately, many people have a poor understanding of their own daily energy (im)balance: the number of calories they consume from food compared with what they expend through physical activity. Accelerometers offer promise as an objective measure of physical activity. In prior work they have been used to estimate energy expenditure and activity type. This work further demonstrates how wireless accelerometers can be used for real-time automatic recognition of physical activity type, intensity, and duration and estimation of energy expenditure. The parameters of the algorithms such as type of classifier/regressor, feature set, window length, signal preprocessing, sensor set utilized and their placement on the human body are selected by performing a set of incremental experiments designed to identify sets of parameters that may balance system usability with robust, real-time performance in low processing power devices such as mobile phones. The algorithms implemented are evaluated using a dataset of examples of 52 activities collected from 20 participants at a gymnasium and a residential home. The algorithms presented here may ultimately allow for the development of mobile phonebased just-in-time interventions to increase self-awareness of physical activity patterns and increases in physical activity levels in real-time during free-living that scale to large populations.

/pdf/using-machine-learning-for-real-time-activity-recognition-46ens73tds.pdf

Using machine learning for real-time activity recognition and estimation of energy expenditure

The ever-growing computation and storage capability of mobile phones have given rise to mobile-centric context recognition systems, which are able to sense and analyze the context of the carrier so as to provide an appropriate level of service. As nonintrusive autonomous sensing and context recognition are desirable characteristics of a personal sensing system; efforts have been made to develop opportunistic sensing techniques on mobile phones. The resulting combination of these approaches has ushered in a new realm of applications, namely opportunistic user context recognition with mobile phones.This article surveys the existing research and approaches towards realization of such systems. In doing so, the typical architecture of a mobile-centric user context recognition system as a sequential process of sensing, preprocessing, and context recognition phases is introduced. The main techniques used for the realization of the respective processes during these phases are described, and their strengths and limitations are highlighted. In addition, lessons learned from previous approaches are presented as motivation for future research. Finally, several open challenges are discussed as possible ways to extend the capabilities of current systems and improve their real-world experience.

A survey on smartphone-based systems for opportunistic user context recognition

User context recognition is one of the important technologies for realizing context aware services. Conventional multi sensor based approach has advantages in that it can generate variety of contexts with less computation resources by using many different sensors. However, such systems tend to be complex and cumbersome and, thus, do not fit in well with mobile environment. In this sense, a single sensor based approach is suitable for mobile environments. In this paper, we show a context inference scheme that realizes a user posture inference with only one acceleration sensor embedded in a mobile handset. Our system automatically detects the sensor position on the user's body and selects the most relevant inference method dynamically. Our experimental results show that the system can infer a user's posture (sitting, standing, walking, and running) with an accuracy of more than 96%.

H. Kurasawa

Papers

Recognizing User Context Using Mobile Handsets with Acceleration Sensors