Gesture elicitation studies represent a popular and resourceful method in HCI to inform the design of intuitive gesture commands, reflective of end-users' behavior, for controlling all kinds of interactive devices, applications, and systems. In the last ten years, an impressive body of work has been published on this topic, disseminating useful design knowledge regarding users' preferences for finger, hand, wrist, arm, head, leg, foot, and whole-body gestures. In this paper, we deliver a systematic literature review of this large body of work by summarizing the characteristics and findings ofN=216gesture elicitation studies subsuming 5,458 participants, 3,625 referents, and 148,340 elicited gestures. We highlight the descriptive, comparative, and generative virtues of our examination to provide practitioners with an effective method to (i) understand how new gesture elicitation studies position in the literature; (ii) compare studies from different authors; and (iii) identify opportunities for new research. We make our large corpus of papers accessible online as a Zotero group library at https://www.zotero.org/groups/2132650/gesture_elicitation_studies.

/pdf/a-systematic-review-of-gesture-elicitation-studies-what-can-1f9x48q8ki.pdf

A Systematic Review of Gesture Elicitation Studies: What Can We Learn from 216 Studies?

Mid-air interaction is a distinct style of natural HCI (Human-Computer Interaction). In mid-air interaction, users make use of their whole body—with a strong focus on hands—and apply gestur...

Empirical Research in Mid-Air Interaction: A Systematic Review

Mid-air interaction involves touchless manipulations of digital content or remote devices, based on sensor tracking of body movements and gestures. There are no established, universal gesture vocabularies for mid-air interactions with digital content or remote devices based on sensor tracking of body movements and gestures. On the contrary, it is widely acknowledged that the identification of appropriate gestures depends on the context of use, thus the identification of mid-air gestures is an important design decision. The method of gesture elicitation is increasingly applied by designers to help them identify appropriate gesture sets for mid-air applications. This paper presents a review of elicitation studies in mid-air interaction based on a selected set of 47 papers published within 2011–2018. It reports on: (1) the application domains of mid-air interactions examined; (2) the level of technological maturity of systems at hand; (3) the gesture elicitation procedure and its variations; (4) the appropriateness criteria for a gesture; (5) participants number and profile; (6) user evaluation methods (of the gesture vocabulary); (7) data analysis and related metrics. This paper confirms that the elicitation method has been applied extensively but with variability and some ambiguity and discusses under-explored research questions and potential improvements of related research.

https://www.mdpi.com/2414-4088/2/4/65/pdf

Gesture Elicitation Studies for Mid-Air Interaction: A Review

This paper presents a prototype of a virtual reality (VR) system to teach the basic concepts of programming called VR‐OCKS. The system is inspired by other visual languages such as Scratch or Kodu, and it works by proposing to the user the resolution of simple puzzles in a 3D environment. Several basic commands to a humanoid character, such as advance or turn, together with control flow structures like iteration and conditional selections, are needed to provide a solution for increasingly difficult challenges. Our aim is to attract users, usually children and teenagers, into the world of programming by taking advantage of the appeal and potential of VR. The use of VR‐OCKS strengthened the spatial orientation and autonomy of the users, in addition to enhancing common sense, creative thinking, and systematic reasoning. In our experiments, VR‐OCKS was accepted by adults and children alike and it showed great potential as an educational tool.

VR-OCKS: A virtual reality game for learning the basic concepts of programming

We present HaptoBend, a novel shape-changing input device providing passive haptic feedback (PHF) for a wide spectrum of objects in virtual reality (VR). Past research in VR shows that PHF increases presence and improves user task performance. However, providing PHF for multiple objects usually requires complex, immobile systems, or multiple props. HaptoBend addresses this problem by allowing users to bend the device into 2D plane-like shapes and multi-surface 3D shapes. We believe HaptoBend's physical approximations of virtual objects can provide realistic haptic feedback through research demonstrating the dominance of human vision over other senses in VR. To test the effectiveness of HaptoBend in matching 2D planar and 3D multi-surface shapes, we conducted an experiment modeled after gesture elicitation studies with 20 participants. High goodness and ease scores show shape-changing passive haptic devices, like HaptoBend, are an effective approach to generalized haptics. Further analysis supports the use of physical approximations for realistic haptic feedback.

Haptobend: shape-changing passive haptic feedback in virtual reality

With the introduction of new input devices, a series of questions have been raised in regard to making user interaction more intuitive - in particular, preferred gestures for different tasks. Our study looks into how to find a gesture set for 3D travel using a multi-touch display and a mid-air device to improve user interaction. We conducted a user study with 30 subjects, concluding that users preferred simple gestures for multi-touch. In addition, we found that multi-touch user legacy carried over mid-Air interaction. Finally, we propose a gesture set for both type of interactions.

Gesture elicitation for 3D travel via multi-touch and mid-Air systems for procedurally generated pseudo-universe

Augmented Reality provides a way to enhance the classroom experience. In particular, student learning about building systems in the fields of Architecture, Civil, and Mechanical Engineering may improve, if visualization outside the classroom is provided. We propose that AR-SKOPE, an application that integrates Building Information Modelling and Augmented Reality may improve learning. This application allows students to visit specific buildings and investigate their various systems with supplementary information using a phone or tablet. We are currently testing our early prototype to conduct a semester-long study.

Integrating Building Information Modeling with Augmented Reality for Interdisciplinary Learning

We propose a 3D Virtual Programming Language to provide an interactive tool for beginners and intermediate students. We believe that the direction of our research will help increase the recruitment and retention of women in CS. We developed an initial prototype and surveyed students to determine the figures that work best. Our results show that it is hard for CS students to provide clear 3D representations for programming concepts in some instances yet we were able to derive some common figures.

Towards a 3D Virtual Programming Language to increase the number of women in computer science education

We present a method of gesture recognition designed to be used with inertial measurement units (IMUs) for 3D user interaction. We provide a drone simulator environment to validate our work. It is important to use motion sensors for user interaction as other devices provide challenges not present in motion-sensors. It is also important since the current field of user interaction is moving towards a multi-modal paradigm and IMUs have been under studied for this field. This work presents the functionality of IMUs with a 6-axis model. We discussed our pilot study and its results, which were satisfactory.

HandMagic: Towards user interaction with inertial measuring units

We present a Multi-Modal Interactive Paint application. Our work is intended to illustrate shortcomings in current multi-modal interaction and to present design strategies to address and alleviate these issues. In particular, from an input perspective use in a regular desktop environment. A serious of challenges are listed and addressed individually with their corresponding strategies in our discussion of design practices for multi-modality. We also identify areas which we will improve for future iterations of similar multi-modal interaction applications due to the findings identified in this paper. These improvements should alleviate shortcomings with our current design and provide further opportunities to research multi-modal interaction.

Jonathan Bernal

Papers

Gesture elicitation for 3D travel via multi-touch and mid-Air systems for procedurally generated pseudo-universe

Integrating Building Information Modeling with Augmented Reality for Interdisciplinary Learning

Towards a 3D Virtual Programming Language to increase the number of women in computer science education

HandMagic: Towards user interaction with inertial measuring units

Towards Multi-modal Interaction with Interactive Paint