Showing papers by "Hans Gellersen published in 2013"

Pursuits: spontaneous interaction with displays based on smooth pursuit eye movement and moving targets

Abstract: Although gaze is an attractive modality for pervasive interactions, the real-world implementation of eye-based interfaces poses significant challenges, such as calibration. We present Pursuits, an innovative interaction technique that enables truly spontaneous interaction with eye-based interfaces. A user can simply walk up to the screen and readily interact with moving targets. Instead of being based on gaze location, Pursuits correlates eye pursuit movements with objects dynamically moving on the interface. We evaluate the influence of target speed, number and trajectory and develop guidelines for designing Pursuits-based interfaces. We then describe six realistic usage scenarios and implement three of them to evaluate the method in a usability study and a field study. Our results show that Pursuits is a versatile and robust technique and that users can interact with Pursuits-based interfaces without prior knowledge or preparation phase.

Pursuit calibration: making gaze calibration less tedious and more flexible

Abstract: Eye gaze is a compelling interaction modality but requires user calibration before interaction can commence. State of the art procedures require the user to fixate on a succession of calibration markers, a task that is often experienced as difficult and tedious. We present pursuit calibration, a novel approach that, unlike existing methods, is able to detect the user's attention to a calibration target. This is achieved by using moving targets, and correlation of eye movement and target trajectory, implicitly exploiting smooth pursuit eye movement. Data for calibration is then only sampled when the user is attending to the target. Because of its ability to detect user attention, pursuit calibration can be performed implicitly, which enables more flexible designs of the calibration task. We demonstrate this in application examples and user studies, and show that pursuit calibration is tolerant to interruption, can blend naturally with applications and is able to calibrate users without their awareness.

Qualitative activity recognition of weight lifting exercises

Abstract: Research on activity recognition has traditionally focused on discriminating between different activities, i.e. to predict which activity was performed at a specific point in time. The quality of executing an activity, the how (well), has only received little attention so far, even though it potentially provides useful information for a large variety of applications. In this work we define quality of execution and investigate three aspects that pertain to qualitative activity recognition: specifying correct execution, detecting execution mistakes, providing feedback on the to the user. We illustrate our approach on the example problem of qualitatively assessing and providing feedback on weight lifting exercises. In two user studies we try out a sensor- and a model-based approach to qualitative activity recognition. Our results underline the potential of model-based assessment and the positive impact of real-time user feedback on the quality of execution.

MotionMA: motion modelling and analysis by demonstration

Abstract: Particularly in sports or physical rehabilitation, users have to perform body movements in a specific manner for the exercises to be most effective. It remains a challenge for experts to specify how to perform such movements so that an automated system can analyse further performances of it. In a user study with 10 participants we show that experts' explicit estimates do not correspond to their performances. To address this issue we present MotionMA, a system that: (1) automatically extracts a model of movements demonstrated by one user, e.g. a trainer, (2) assesses the performance of other users repeating this movement in real time, and (3) provides real-time feedback on how to improve their performance. We evaluated the system in a second study in which 10 other participants used the system to demonstrate arbitrary movements. Our results demonstrate that MotionMA is able to extract an accurate movement model to spot mistakes and variations in movement execution.

SideWays: a gaze interface for spontaneous interaction with situated displays

Abstract: Eye gaze is compelling for interaction with situated displays as we naturally use our eyes to engage with them. In this work we present SideWays, a novel person-independent eye gaze interface that supports spontaneous interaction with displays: users can just walk up to a display and immediately interact using their eyes, without any prior user calibration or training. Requiring only a single off-the-shelf camera and lightweight image processing, SideWays robustly detects whether users attend to the centre of the display or cast glances to the left or right. The system supports an interaction model in which attention to the central display is the default state, while "sidelong glances" trigger input or actions. The robustness of the system and usability of the interaction model are validated in a study with 14 participants. Analysis of the participants' strategies in performing different tasks provides insights on gaze control strategies for design of SideWays applications.

Enclosed: a component-centric interface for designing prototype enclosures

Abstract: This paper explores the problem of designing enclosures (or physical cases) that are needed for prototyping electronic devices. We present a novel interface that uses electronic components as handles for designing the 3D shape of the enclosure. We use the .NET Gadgeteer platform as a case study of this problem, and implemented a proof-of-concept system for designing enclosures for Gadgeteer components. We show examples of enclosures designed and fabricated with our system.

EyeContext: recognition of high-level contextual cues from human visual behaviour

Abstract: In this work we present EyeContext, a system to infer high-level contextual cues from human visual behaviour. We conducted a user study to record eye movements of four participants over a full day of their daily life, totalling 42.5 hours of eye movement data. Participants were asked to self-annotate four non-mutually exclusive cues: social (interacting with somebody vs. no interaction), cognitive (concentrated work vs. leisure), physical (physically active vs. not active), and spatial (inside vs. outside a building). We evaluate a proof-of-concept EyeContext system that combines encoding of eye movements into strings and a spectrum string kernel support vector machine (SVM) classifier. Our results demonstrate the large information content available in long-term human visual behaviour and opens up new venues for research on eye-based behavioural monitoring and life logging.

AffectCam: arousal- augmented sensecam for richer recall of episodic memories

Abstract: This paper describes the design and evaluation of AffectCam, a wearable system integrating SenseCam and BodyMedia SenseWear for capturing galvanic skin response as a measure of bodily arousal. AffectCam's algorithms use arousal as a filtering mechanism for selecting the most personally relevant photos captured during people's ordinary daily life, i.e. high arousal photos. We discuss initial findings showing that emotional arousal does improve the quality of memory recall associated with emotionally arousing events. In particular, the high arousal photos support richer recall of episodic memories than low arousal ones, i.e. over 50% improvement. We also consider how various phenomenological characteristics of autobiographical memories such as event, emotions, thoughts, place and time are differently cued by the AffectCam.

Eye Pull, Eye Push: Moving Objects between Large Screens and Personal Devices with Gaze and Touch

Abstract: Previous work has validated the eyes and mobile input as a viable approach for pointing at, and selecting out of reach objects. This work presents Eye Pull, Eye Push, a novel interaction concept for content transfer between public and personal devices using gaze and touch. We present three techniques that enable this interaction: Eye Cut & Paste, Eye Drag & Drop, and Eye Summon & Cast. We outline and discuss several scenarios in which these techniques can be used. In a user study we found that participants responded well to the visual feedback provided by Eye Drag & Drop during object movement. In contrast, we found that although Eye Summon & Cast significantly improved performance, participants had difficulty coordinating their hands and eyes during interaction.

Pursuits: eye-based interaction with moving targets

Abstract: Eye-based interaction has commonly been based on estimation of eye gaze direction, to locate objects for interaction. We introduce Pursuits, a novel and very different eye tracking method that instead is based on following the trajectory of eye movement and comparing this with trajectories of objects in the field of view. Because the eyes naturally follow the trajectory of moving objects of interest, our method is able to detect what the user is looking at, by matching eye movement and object movement. We illustrate Pursuits with three applications that demonstrate how the method facilitates natural interaction with moving targets.

AutoBAP: Automatic Coding of Body Action and Posture Units from Wearable Sensors

Abstract: Manual annotation of human body movement is an integral part of research on non-verbal communication and computational behaviour analysis but also a very time-consuming and tedious task. In this paper we present AutoBAP, a system that automates the coding of bodily expressions according to the body action and posture (BAP) coding scheme. Our system takes continuous body motion and gaze behaviour data as its input. The data is recorded using a full body motion tracking suit and a wearable eye tracker. From the data our system automatically generates a labelled XML file that can be visualised and edited with off-the-shelf video annotation tools. We evaluate our system in a laboratory-based user study with six participants performing scripted sequences of 184 actions. Results from the user study show that our prototype system is able to annotate 172 out of the 274 labels of the full BAP coding scheme with good agreement with a manual annotator (Cohen's kappa > 0.6).

A cross-device drag-and-drop technique

Abstract: Many interactions naturally extend across smart-phones and devices with larger screens. Indeed, data might be received on the mobile but more conveniently processed with an application on a larger device, or vice versa. Such interactions require spontaneous data transfer from a source location on one screen to a target location on the other device. We introduce a cross-device Drag-and-Drop technique to facilitate these interactions involving multiple touchscreen devices, with minimal effort for the user. The technique is a two-handed gesture, where one hand is used to suitably align the mobile phone with the larger screen, while the other is used to select and drag an object between devices and choose which application should receive the data.

How groups of users associate wireless devices

Abstract: Group association, the process of connecting a group of devices, opens up new opportunities for users to spontaneously share resources. Research has shown numerous techniques and protocols for group association; however, what people intuitively do to associate a group of devices remains an open question. We contribute a study of eliciting device association techniques from groups of non-technical people. In all, we collected and analysed 496 techniques from 61 participants. Our results show that mobility and physicality of devices influence how people perceive groups association. We present a complete set of user-defined techniques with subjective ratings and popularity scores. We examined people's rationale and the effects of different device form factors. We analysed the techniques based on the roles that users assume with respect to device association. Our findings draw out insights from the perspective of users for design of group association.

Personal clipboards for individual copy-and-paste on shared multi-user surfaces

Abstract: Clipboards are omnipresent on today's personal computing platforms They provide copy-and-paste functionalities that let users easily reorganize information and quickly transfer data across applications In this work, we introduce personal clipboards to multi-user surfaces Personal clipboards enable individual and independent copy-and-paste operations, in the presence of multiple users concurrently sharing the same direct-touch interface As common surface computing platforms do not distinguish touch input of different users, we have developed clipboards that leverage complementary personalization strategies Specifically, we have built a context menu clipboard based on implicit user identification of every touch, a clipboard based on personal subareas dynamically placed on the surface, and a handheld clipboard based on integration of personal devices for surface interaction In a user study, we demonstrate the effectiveness of personal clipboards for shared surfaces, and show that different personalization strategies enable clipboards, albeit with different impacts on interaction characteristics

Cooperative augmentation of mobile smart objects with projected displays

Abstract: Sensors, processors, and radios can be integrated invisibly into objects to make them smart and sensitive to user interaction, but feedback is often limited to beeps, blinks, or buzzes. We propose to redress this input-output imbalance by augmentation of smart objects with projected displays, that—unlike physical displays—allow seamless integration with the natural appearance of an object. In this article, we investigate how, in a ubiquitous computing world, smart objects can acquire and control a projection. We consider that projectors and cameras are ubiquitous in the environment, and we develop a novel conception and system that enables smart objects to spontaneously associate with projector-camera systems for cooperative augmentation. Projector-camera systems are conceived as generic, supporting standard computer vision methods for different appearance cues, and smart objects provide a model of their appearance for method selection at runtime, as well as sensor observations to constrain the visual detection process. Cooperative detection results in accurate location and pose of the object, which is then tracked for visual augmentation in response to display requests by the smart object. In this article, we define the conceptual framework underlying our approach; report on computer vision experiments that give original insight into natural appearance-based detection of everyday objects; show how object sensing can be used to increase speed and robustness of visual detection; describe and evaluate a fully implemented system; and describe two smart object applications to illustrate the system's cooperative augmentation process and the embodied interactions it enables with smart objects.

Eye drop: an interaction concept for gaze-supported point-to-point content transfer

Abstract: The shared displays in our environment contain content that we desire. Furthermore, we often acquire content for a specific purpose, i.e., the acquisition of a phone number to place a call. We have developed a content transfer concept, Eye Drop. Eye Drop provides techniques that allow fluid content acquisition, transfer from shared displays, and local positioning on personal devices using gaze combined with manual input. The eyes naturally focus on content we desire. Our techniques use gaze to point remotely, removing the need for explicit pointing on the user's part. A manual trigger from a personal device confirms selection. Transfer is performed using gaze or manual input to smoothly transition content to a specific location on a personal device. This work demonstrates how techniques can be applied to acquire and apply actions to content through a natural sequence of interaction. We demonstrate a proof of concept prototype through five implemented application scenarios.

QRS complex detection using centered Cumulative Sums of Squares

Abstract: This paper proposes a new framework for accurate detection of the QRS complex based on centered Cumulative Sums of Squares (CUSUM). This detection of the QRS complex is very important for many health applications. To accurately detect the QRS wave, the proposed framework first uses filters to reduce the frequency of big T and P waves. Then, it applies the CUSUM and differential operation to produce a set of event vectors and uses a fixed threshold to detect the QRS candidates. Then, a width feature is used to discriminate the non-QRS complex from the QRS complex event. For the purpose of evaluation, the standard ECG MIT-BIH database has been used. The evaluation results show that our method has an accuracy of 99.66%.

Technical framework supporting a cross-device drag-and-drop technique

Abstract: We present the technical framework supporting a cross-device Drag-and-Drop technique designed to facilitate interactions involving multiple touchscreen devices. This technique supports users that need to transfer information received or produced on one device to another device which might be more suited to process it. Furthermore, it does not require any additional instrumentation. The technique is a two-handed gesture where one hand is used to suitably align the mobile phone with the larger screen, while the other is used to select and drag an object from one device to the other where it can be applied directly onto a target application. We describe the implementation of the framework that enables spontaneous data-transfer between a mobile device and a desktop computer.