Providing accurate and opportune information on people's activities and behaviors is one of the most important tasks in pervasive computing. Innumerable applications can be visualized, for instance, in medical, security, entertainment, and tactical scenarios. Despite human activity recognition (HAR) being an active field for more than a decade, there are still key aspects that, if addressed, would constitute a significant turn in the way people interact with mobile devices. This paper surveys the state of the art in HAR based on wearable sensors. A general architecture is first presented along with a description of the main components of any HAR system. We also propose a two-level taxonomy in accordance to the learning approach (either supervised or semi-supervised) and the response time (either offline or online). Then, the principal issues and challenges are discussed, as well as the main solutions to each one of them. Twenty eight systems are qualitatively evaluated in terms of recognition performance, energy consumption, obtrusiveness, and flexibility, among others. Finally, we present some open problems and ideas that, due to their high relevance, should be addressed in future research.

A Survey on Human Activity Recognition using Wearable Sensors

An apposite and eminently readable reference for all behavioral science research and development

Encyclopedia of Statistics in Behavioral Science

Physical activity recognition using embedded sensors has enabled many context-aware applications in different areas, such as healthcare Initially, one or more dedicated wearable sensors were used for such applications However, recently, many researchers started using mobile phones for this purpose, since these ubiquitous devices are equipped with various sensors, ranging from accelerometers to magnetic field sensors In most of the current studies, sensor data collected for activity recognition are analyzed offline using machine learning tools However, there is now a trend towards implementing activity recognition systems on these devices in an online manner, since modern mobile phones have become more powerful in terms of available resources, such as CPU, memory and battery The research on offline activity recognition has been reviewed in several earlier studies in detail However, work done on online activity recognition is still in its infancy and is yet to be reviewed In this paper, we review the studies done so far that implement activity recognition systems on mobile phones and use only their on-board sensors We discuss various aspects of these studies Moreover, we discuss their limitations and present various recommendations for future research

https://ris.utwente.nl/ws/files/6518323/sensors-15-02059.pdf

A survey of online activity recognition using mobile phones

Integrating the various embedded devices and systems in our environment enables an Internet of Things (IoT) for a smart city. The IoT will generate tremendous amount of data that can be leveraged for safety, efficiency, and infotainment applications and services for city residents. The management of this voluminous data through its lifecycle is fundamental to the realization of smart cities. Therefore, in contrast to existing surveys on smart cities we provide a data-centric perspective, describing the fundamental data management techniques employed to ensure consistency, interoperability, granularity, and reusability of the data generated by the underlying IoT for smart cities. Essentially, the data lifecycle in a smart city is dependent on tightly coupled data management with cross-cutting layers of data security and privacy, and supporting infrastructure. Therefore, we further identify techniques employed for data security and privacy, and discuss the networking and computing technologies that enable smart cities. We highlight the achievements in realizing various aspects of smart cities, present the lessons learned, and identify limitations and research challenges.

Smart Cities: A Survey on Data Management, Security, and Enabling Technologies

Activity detection and classification are very important for autonomous monitoring of humans for applications, including assistive living, rehabilitation, and surveillance. Wearable sensors have found wide-spread use in recent years due to their ever-decreasing cost, ease of deployment and use, and ability to provide continuous monitoring as opposed to sensors installed at fixed locations. Since many smart phones are now equipped with a variety of sensors, such as accelerometer, gyroscope, and camera, it has become more feasible to develop activity monitoring algorithms employing one or more of these sensors with increased accessibility. We provide a complete and comprehensive survey on activity classification with wearable sensors, covering a variety of sensing modalities, including accelerometer, gyroscope, pressure sensors, and camera- and depth-based systems. We discuss differences in activity types tackled by this breadth of sensing modalities. For example, accelerometer, gyroscope, and magnetometer systems have a history of addressing whole body motion or global type activities, whereas camera systems provide the context necessary to classify local interactions, or interactions of individuals with objects. We also found that these single sensing modalities laid the foundation for hybrid works that tackle a mix of global and local interaction-type activities. In addition to the type of sensors and type of activities classified, we provide details on each wearable system that include on-body sensor location, employed learning approach, and extent of experimental setup. We further discuss where the processing is performed, i.e., local versus remote processing, for different systems. This is one of the first surveys to provide such breadth of coverage across different wearable sensor systems for activity classification.

A Survey on Activity Detection and Classification Using Wearable Sensors

Mobile phones' increasing ubiquity has created many opportunities for personal context sensing. Personal activity is an important part of a user's context, and automatically recognizing it is vital for health and fitness monitoring applications. Recording a stream of activity data enables monitoring patients with chronic conditions affecting ambulation and motion, as well as those undergoing rehabilitation treatments. Modern mobile phones are powerful enough to perform activity classification in real time, but they typically use a static classifier that is trained in advance or require the user to manually add training data after the application is on his/her device. This paper investigates ways of automatically augmenting activity classifiers after they are deployed in an application. It compares active learning and three different semi-supervised learning methods, self-learning, En-Co-Training, and democratic co-learning, to determine which show promise for this purpose. The results show that active learning, En-Co-Training, and democratic co-learning perform well when the initial classifier's accuracy is low (75–80%). When the initial accuracy is already high (90%), these methods are no longer effective, but they do not hurt the accuracy either. Overall, active learning gave the highest improvement, but democratic co-learning was almost as good and does not require user interaction. Thus, democratic co-learning would be the best choice for most applications, since it would significantly increase the accuracy for initial classifiers that performed poorly.

/pdf/improving-activity-classification-for-health-applications-on-fc8usvlzmr.pdf

Improving activity classification for health applications on mobile devices using active and semi-supervised learning

An important tool for evaluating the health of patients who suffer from mobility-affecting chronic diseases such as MS, Parkinson’s, and Muscular Dystrophy is assessment of how much they walk. Ambulation is a mobility monitoring system that uses Android and Nokia N95 mobile phones to automatically detect the user’s mobility mode. The user’s only required interaction with the phone is turning it on and keeping it with him/her throughout the day, with the intention that it could be used as his/her everyday mobile phone for voice, data, and other applications, while Ambulation runs in the background. The phone uploads the collected mobility and location information to a server and a secure, intuitive web-based visualization of the data is available to the user and any family, friends or caregivers whom they authorize, allowing them to identify trends in their mobility and measure progress over time and in response to varying treatments.

Ambulation: a tool for monitoring mobility patterns over time using mobile phones

/pdf/ambulation-a-tool-for-monitoring-mobility-patterns-over-time-1fqm2lm7u1.pdf

Ambulation: A Tool for Monitoring Mobility Patterns over Time Using Mobile Phones

We present ohmage, a mobile to web platform that records, analyzes, and visualizes data from both prompted experience samples entered by the user, as well as continuous streams of data passively collected from sensors onboard the mobile device. ohmage has been used in a number of research health studies. Key challenges in these deployments are engaging participants to sustain data collection in long-lived campaigns, conserving battery power, and extracting accurate inferences from the collected streams. To address these challenges, we have incorporated feedback from hundreds of behavioral and technology researchers, focus group participants, and end-users of the system in an iterative design process. We summarize this rich feedback, and present the resulting system.

/pdf/ohmage-an-open-mobile-system-for-activity-and-experience-28li0zi75h.pdf

ohmage: An open mobile system for activity and experience sampling

Participatory sensing (PS) is a distributed data collection and analysis approach where individuals, acting alone or in groups, use their personal mobile devices to systematically explore interesting aspects of their lives and communities [Burke et al. 2006]. These mobile devices can be used to capture diverse spatiotemporal data through both intermittent self-report and continuous recording from on-board sensors and applications. Ohmage (http://ohmage.org) is a modular and extensible open-source, mobile to Web PS platform that records, stores, analyzes, and visualizes data from both prompted self-report and continuous data streams. These data streams are authorable and can dynamically be deployed in diverse settings. Feedback from hundreds of behavioral and technology researchers, focus group participants, and end users has been integrated into ohmage through an iterative participatory design process. Ohmage has been used as an enabling platform in more than 20 independent projects in many disciplines. We summarize the PS requirements, challenges and key design objectives learned through our design process, and ohmage system architecture to achieve those objectives. The flexibility, modularity, and extensibility of ohmage in supporting diverse deployment settings are presented through three distinct case studies in education, health, and clinical research.

http://ohmage.org/static/TIST-a38-ohmage.pdf

Brent Longstaff

Papers

Improving activity classification for health applications on mobile devices using active and semi-supervised learning

Ambulation: a tool for monitoring mobility patterns over time using mobile phones

Ambulation: A Tool for Monitoring Mobility Patterns over Time Using Mobile Phones

ohmage: An open mobile system for activity and experience sampling

Ohmage: A General and Extensible End-to-End Participatory Sensing Platform