Proceedings ArticleDOI
SoundSense: scalable sound sensing for people-centric applications on mobile phones
Hong Lu,Wei Pan,Nicholas D. Lane,Tanzeem Choudhury,Andrew T. Campbell +4 more
- pp 165-178
TLDR
This paper proposes SoundSense, a scalable framework for modeling sound events on mobile phones that represents the first general purpose sound sensing system specifically designed to work on resource limited phones and demonstrates that SoundSense is capable of recognizing meaningful sound events that occur in users' everyday lives.Abstract:
Top end mobile phones include a number of specialized (e.g., accelerometer, compass, GPS) and general purpose sensors (e.g., microphone, camera) that enable new people-centric sensing applications. Perhaps the most ubiquitous and unexploited sensor on mobile phones is the microphone - a powerful sensor that is capable of making sophisticated inferences about human activity, location, and social events from sound. In this paper, we exploit this untapped sensor not in the context of human communications but as an enabler of new sensing applications. We propose SoundSense, a scalable framework for modeling sound events on mobile phones. SoundSense is implemented on the Apple iPhone and represents the first general purpose sound sensing system specifically designed to work on resource limited phones. The architecture and algorithms are designed for scalability and Soundsense uses a combination of supervised and unsupervised learning techniques to classify both general sound types (e.g., music, voice) and discover novel sound events specific to individual users. The system runs solely on the mobile phone with no back-end interactions. Through implementation and evaluation of two proof of concept people-centric sensing applications, we demostrate that SoundSense is capable of recognizing meaningful sound events that occur in users' everyday lives.read more
Citations
More filters
Pattern Recognition and Machine Learning
TL;DR: Probability distributions of linear models for regression and classification are given in this article, along with a discussion of combining models and combining models in the context of machine learning and classification.
Journal ArticleDOI
A survey of mobile phone sensing
Nicholas D. Lane,Emiliano Miluzzo,Hong Lu,Daniel Peebles,Tanzeem Choudhury,Andrew T. Campbell +5 more
TL;DR: This article surveys existing mobile phone sensing algorithms, applications, and systems, and discusses the emerging sensing paradigms, and formulates an architectural framework for discussing a number of the open issues and challenges emerging in the new area ofMobile phone sensing research.
Journal ArticleDOI
A tutorial on human activity recognition using body-worn inertial sensors
TL;DR: In this paper, the authors provide a comprehensive hands-on introduction for newcomers to the field of human activity recognition using on-body inertial sensors and describe the concept of an Activity Recognition Chain (ARC) as a general-purpose framework for designing and evaluating activity recognition systems.
A Tutorial on Human Activity Recognition Using Body-Worn
Andreas Bulling,Ulf Blanke +1 more
TL;DR: This tutorial aims to provide a comprehensive hands-on introduction for newcomers to the field of human activity recognition using on-body inertial sensors and describes the concept of an Activity Recognition Chain (ARC) as a general-purpose framework for designing and evaluating activity recognition systems.
Patent
Intuitive computing methods and systems
TL;DR: A smart phone senses audio, imagery, and/or other stimulus from a user's environment, and acts autonomously to fulfill inferred or anticipated user desires as discussed by the authors, and can apply more or less resources to an image processing task depending on how successfully the task is proceeding or based on the user's apparent interest in the task.
References
More filters
Journal Article
Auditory Context Awareness via Wearable Computing
TL;DR: A system for obtaining environmental context through audio for applications and user interfaces that relies on unsupervised training for segmentation of sound scenes and detects and classifies events and scenes using a HMM framework.
Proceedings Article
Voice signatures
TL;DR: Two approaches for extracting speaker traits are investigated: the first focuses on general acoustic and prosodic features, the second on the choice of words used by the speaker, showing that voice signatures are of practical interest in real-world applications.
Dissertation
Sensing and modeling human networks
Tanzeem Choudhury,Alex Pentland +1 more
TL;DR: This thesis develops a computational framework for learning the interaction structure and dynamics automatically from the sociometer data, which converts low-level sensor data into measures that can be used to learn socially relevant aspects of people's interactions.
Proceedings ArticleDOI
Peopletones: a system for the detection and notification of buddy proximity on mobile phones
TL;DR: An algorithm for detecting proximity, techniques for reducing sensor noise and power consumption, and a method for generating peripheral cues are contributed, enabling the study of how people respond to peripheral cues in the wild.
Proceedings Article
Context Awareness using Environmental Noise Classification
Ling Ma,Dan Smith,Ben Milner +2 more
TL;DR: The approach for automatically sensing and recognising noise from typical environments of daily life, such as office, car and city street, is described and the hidden Markov model based noise classifier is presented.