Showing papers by "Jamie A. Ward published in 2012"

Multimodal recognition of reading activity in transit using body-worn sensors

Abstract: Reading is one of the most well-studied visual activities Vision research traditionally focuses on understanding the perceptual and cognitive processes involved in reading In this work we recognize reading activity by jointly analyzing eye and head movements of people in an everyday environment Eye movements are recorded using an electrooculography (EOG) system; body movements using body-worn inertial measurement units We compare two approaches for continuous recognition of reading: String matching (STR) that explicitly models the characteristic horizontal saccades during reading, and a support vector machine (SVM) that relies on 90 eye movement features extracted from the eye movement data We evaluate both methods in a study performed with eight participants reading while sitting at a desk, standing, walking indoors and outdoors, and riding a tram We introduce a method to segment reading activity by exploiting the sensorimotor coordination of eye and head movements during reading Using person-independent training, we obtain an average precision for recognizing reading of 889p (recall 723p) using STR and of 877p (recall 879p) using SVM over all participants We show that the proposed segmentation scheme improves the performance of recognizing reading events by more than 24p Our work demonstrates that the joint analysis of eye and body movements is beneficial for reading recognition and opens up discussion on the wider applicability of a multimodal recognition approach to other visual and physical activities