As computers become more pervasive in society, facilitating natural human---computer interaction (HCI) will have a positive impact on their use. Hence, there has been growing interest in the development of new approaches and technologies for bridging the human---computer barrier. The ultimate aim is to bring HCI to a regime where interactions with computers will be as natural as an interaction between humans, and to this end, incorporating gestures in HCI is an important research area. Gestures have long been considered as an interaction technique that can potentially deliver more natural, creative and intuitive methods for communicating with our computers. This paper provides an analysis of comparative surveys done in this area. The use of hand gestures as a natural interface serves as a motivating force for research in gesture taxonomies, its representations and recognition techniques, software platforms and frameworks which is discussed briefly in this paper. It focuses on the three main phases of hand gesture recognition i.e. detection, tracking and recognition. Different application which employs hand gestures for efficient interaction has been discussed under core and advanced application domains. This paper also provides an analysis of existing literature related to gesture recognition systems for human computer interaction by categorizing it under different key parameters. It further discusses the advances that are needed to further improvise the present hand gesture recognition systems for future perspective that can be widely used for efficient human computer interaction. The main goal of this survey is to provide researchers in the field of gesture based HCI with a summary of progress achieved to date and to help identify areas where further research is needed.

/pdf/vision-based-hand-gesture-recognition-for-human-computer-3dczparvi2.pdf

Vision based hand gesture recognition for human computer interaction: a survey

While applications for mobile devices have become extremely important in the last few years, little public information exists on mobile application usage behavior. We describe a large-scale deployment-based research study that logged detailed application usage information from over 4,100 users of Android-powered mobile devices. We present two types of results from analyzing this data: basic descriptive statistics and contextual descriptive statistics. In the case of the former, we find that the average session with an application lasts less than a minute, even though users spend almost an hour a day using their phones. Our contextual findings include those related to time of day and location. For instance, we show that news applications are most popular in the morning and games are at night, but communication applications dominate through most of the day. We also find that despite the variety of apps available, communication applications are almost always the first used upon a device's waking from sleep. In addition, we discuss the notion of a virtual application sensor, which we used to collect the data.

/pdf/falling-asleep-with-angry-birds-facebook-and-kindle-a-large-j46nmw401v.pdf

Falling asleep with Angry Birds, Facebook and Kindle: a large scale study on mobile application usage

Body posture and finger pointing are a natural modality for human-machine interaction, but first the system must know what it's seeing.

Vision-based hand-gesture applications

Notifications are a core feature of mobile phones. They inform users about a variety of events. Users may take immediate action or ignore them depending on the importance of a notification as well as their current context. The nature of notifications is manifold, applications use them both sparsely and frequently. In this paper we present the first large-scale analysis of mobile notifications with a focus on users' subjective perceptions. We derive a holistic picture of notifications on mobile phones by collecting close to 200 million notifications from more than 40,000 users. Using a data-driven approach, we break down what users like and dislike about notifications. Our results reveal differences in importance of notifications and how users value notifications from messaging apps as well as notifications that include information about people and events. Based on these results we derive a number of findings about the nature of notifications and guidelines to effectively use them.

/pdf/large-scale-assessment-of-mobile-notifications-3ayexh71go.pdf

Large-scale assessment of mobile notifications

New high-frequency multimodal data collection technologies and machine learning analysis techniques could offer new insights into learning, especially when students have the opportunity to generate unique, personalized artifacts, such as computer programs, robots, and solutions engineering challenges. To date most of the work on learning analytics and educational data mining has been focused on online courses and cognitive tutors, both of which provide a high degree of structure to the tasks, and are restricted to interactions that occur in front of a computer screen. In this paper, we argue that multimodal learning analytics can offer new insights into students’ learning trajectories in more complex and open-ended learning environments. We present several examples of this work and its educational application.

/pdf/multimodal-learning-analytics-and-education-data-mining-11tv8qs17p.pdf

Multimodal Learning Analytics and Education Data Mining: Using computational technologies to measure complex learning tasks

In many applications today user interaction is moving away from mouse and pens and is becoming pervasive and much more physical and tangible. New emerging interaction technologies allow developing and experimenting with new interaction methods on the long way to providing intuitive human computer interaction. In this paper, we aim at recognizing gestures to interact with an application and present the design and evaluation of our sensor-based gesture recognition. As input device we employ the Wii-controller (Wiimote) which recently gained much attention world wide. We use the Wiimote's acceleration sensor independent of the gaming console for gesture recognition. The system allows the training of arbitrary gestures by users which can then be recalled for interacting with systems like photo browsing on a home TV. The developed library exploits Wii-sensor data and employs a hidden Markov model for training and recognizing user-chosen gestures. Our evaluation shows that we can already recognize gestures with a small number of training samples. In addition to the gesture recognition we also present our experiences with the Wii-controller and the implementation of the gesture recognition. The system forms the basis for our ongoing work on multimodal intuitive media browsing and are available to other researchers in the field.

/pdf/gesture-recognition-with-a-wii-controller-2l00c7gl9r.pdf

Gesture recognition with a Wii controller

This article reports on a preliminary study, which investigates if vibration and speech feedback can be used in order to make a digital map on a touch screen device more accessible. We test if vibration feedback combined with speech, triggered as the finger moves over relevant map objects, works to make sense of the map content. The study results indicate that it is indeed possible to get a basic overview of the map layout even if a person does not have access to the visual presentation. In the conclusions the interaction problems are indentified and suggestions for future improvements are given.

TouchOver map: audio-tactile exploration of interactive maps

Pedestrian navigation systems are becoming popular but the currently dominant audio-visual interaction can have drawbacks. Tactile feedback is studied as a solution, but currently only available as research prototypes. With the PocketNavigator we propose a demonstrator that adds tactile feedback to a simple but robust map-based navigation system that runs on any Android Smartphone. Users can leave the device in the pocket, while being guided non-visually through vibration cues. Like a compass we "point at" the next waypoint by encoding its direction and distance in vibration patterns. As an advantage over previous approaches it allows giving continuous feedback instead of isolated turning instructions and it can be realized without custom-built tactile displays. Preliminary results from a field study show that pedestrian can effectively use this Tactile Compass to reach a destination without turn-by-turn instructions. Integrated into the PocketNavigator we can now deploy it at the Android Market to evaluate the Tactile Compass with a wide range of users.

PocketNavigator: vibro-tactile waypoint navigation for everyday mobile devices

Experiments are a cornerstone of HCI research. Mobile distribution channels such as Apple's App Store and Google's Android Market have created the opportunity to bring experiments to the end user. Hardly any experience exists on how to conduct such experiments successfully. This article reports on five experiments that were conducted by publishing Apps in the Android Market. The Apps are freely available and have been installed more than 30,000 times. The outcomes of the experiments range from failure to valuable insights. Based on these outcomes, the authors identified factors that account for the success of experiments using mobile application stores. When generalizing findings it must be considered that smartphone users are a non-representative sample of the world's population. Most participants can be obtained by informing users about the study when the App had been started for the first time. Because Apps are often used for a short time only, data should be collected as early as possible. To collect valuable qualitative feedback other channels than user comments and email have to be used. Finally, the interpretation of collected data has to consider unpredicted usage patterns to provide valid conclusions.

/pdf/my-app-is-an-experiment-experience-from-user-studies-in-7snhlotvwb.pdf

My App is an Experiment: Experience from User Studies in Mobile App Stores

Today's smartphones will issue a notification immediately after an event occurs, repeating unanswered notifications in fixed time intervals. The disadvantage of this issue-and-repeat strategy is that notifications can appear in inconvenient situations and thus are perceived as annoying and interrupting. The authors study the mobile context as inferred through a phone's sensors for both answered and ignored notifications. They conducted a large-scale, longitudinal study via the Google Play store and observed 6,581 notifications from 79 different users over 76 days. Their derived model can predict opportune moments to issue notifications with approximately 77 percent accuracy. Their findings could lead to intelligent strategies to issue unobtrusive notifications on today's smartphones at no extra cost. This article is part of a special issue on managing attention.

Benjamin Poppinga

Papers

Gesture recognition with a Wii controller

TouchOver map: audio-tactile exploration of interactive maps

PocketNavigator: vibro-tactile waypoint navigation for everyday mobile devices

My App is an Experiment: Experience from User Studies in Mobile App Stores

Sensor-Based Identification of Opportune Moments for Triggering Notifications