Display devices on and around the body such as smartwatches, head-mounted displays or tablets enable users to interact on the go. However, diverging input and output fidelities of these devices can lead to interaction seams that can inhibit efficient mobile interaction, when users employ multiple devices at once. We present MultiFi, an interactive system that combines the strengths of multiple displays and overcomes the seams of mobile interaction with widgets distributed over multiple devices. A comparative user study indicates that combined head-mounted display and smartwatch interfaces can outperform interaction with single wearable devices.

/pdf/multifi-multi-fidelity-interaction-with-displays-on-and-2rcjyf4p2f.pdf

MultiFi: Multi Fidelity Interaction with Displays On and Around the Body

A pressure sensing based single-handed interaction model is presented in this paper. Unlike traditional desktop GUI model, mobile UI model has not been established yet. For example, Apple iPhone proposed "Pinch" operation, which use two fingers to zoom-in and zoom-out objects. However, in a today's hand-held situation, manipulation methods using two fingers are not always good solution because they require two hands in order to hold the device itself in most cases. We propose a single-handed UI scheme "GraspZoom": multi-state input model using pressure sensing. Force Sensitive Resistor (FSR) attached on backside of a mobile phone was employed in order to evaluate effectiveness of pressure based control model. We also describe example applications which enable intuitive and continuous zooming and scrolling. By using tiny thumb gesture input along with this pressure sensing method, bi-directional operations (e.g., zoom-in and -out) are also achieved.

GraspZoom: Zooming and scrolling control model for single-handed mobile interaction

We propose RotoSwype, a technique for word-gesture typing using the orientation of a ring worn on the index finger. RotoSwype enables one-handed text-input without encumbering the hand with a device, a desirable quality in many scenarios, including virtual or augmented reality. The method is evaluated using two arm positions: with the hand raised up with the palm parallel to the ground; and with the hand resting at the side with the palm facing the body. A five-day study finds both hand positions achieved speeds of at least 14 words-per-minute (WPM) with uncorrected error rates near 1%, outperforming previous comparable techniques.

/pdf/rotoswype-word-gesture-typing-using-a-ring-20ytrdo2wm.pdf

RotoSwype: Word-Gesture Typing using a Ring

Raycasting is the most common target pointing technique in virtual reality environments. However, performance on small and distant targets is impacted by the accuracy of the pointing device and the user's motor skills. Current pointing facilitation techniques are currently only applied in the context of the virtual hand, i.e. for targets within reach. We propose enhancements to Raycasting: filtering the ray, and adding a controllable cursor on the ray to select the nearest target. We describe a series of studies for the design of the visual feedforward, filtering technique, as well as a comparative study between different 3D pointing techniques. Our results show that highlighting the nearest target is one of the most efficient visual feedforward technique. We also show that filtering the ray reduces error rate in a drastic way. Finally we show the benefits of RayCursor compared to Raycasting and another technique from the literature.

https://hal.inria.fr/hal-02015844/document

RayCursor: A 3D Pointing Facilitation Technique based on Raycasting

In Cinematic Virtual Reality (CVR), the viewer of an omnidirectional movie can freely choose the viewing direction when watching a movie. Therefore, traditional techniques in filmmaking for guiding the viewers’ attention cannot be adapted directly to CVR. Practices such as panning or changing the frame are no longer defined by the filmmaker; rather it is the viewer who decides where to look. In some stories, it is necessary to show certain details to the viewer, which should not be missed. At the same time, the freedom of the viewer to look around in the scene should not be destroyed. Therefore, techniques are needed which guide the attention of the spectator to visual information in the scene. Attention guiding also has the potential to improve the general viewing experience, since viewers will be less afraid to miss something when watching an omnidirectional movie where attention-guiding techniques have been applied. In recent years, there has been a lot of research about attention guiding in images, movies, virtual reality, augmented reality and also in CVR. We classify these methods and offer a taxonomy for attention-guiding methods. Discussing the different characteristics, we elaborate the advantages and disadvantages, give recommendations for use cases and apply the taxonomy to several examples of guiding methods.

https://www.mdpi.com/2414-4088/3/1/19/pdf

Guidance in Cinematic Virtual Reality-Taxonomy, Research Status and Challenges

2D or 3D gesture commands are still not routinely adopted, despite the technological advances for tracking gestures. The fact that gesture commands are not self-revealing is a bottleneck for this adoption. Guiding novice users is therefore crucial in order to reveal what commands are available and how to trigger them. However guiding systems are mainly designed in an ad hoc manner. Even if isolated design characteristics exist, they concentrate on a limited number of guidance aspects. We hence present a design space that unifies and completes these studies by providing a coherent set of issues for designing the behavior of a guiding system. We distinguish Feedback and Feedforward and consider four questions: When, What, How and Where. In order to leverage efficient use of our design space, we provide an online tool and illustrate with scenarios how practitioners can use it.

/pdf/designing-guiding-systems-for-gesture-based-interaction-1n9zm3ui9g.pdf

Designing guiding systems for gesture-based interaction

Dynamic symbolic in-air hand gestures are an increasingly popular means of interaction with smart environments However, novices need to know what commands are available and which gesture to execute in order to trigger these commands We propose to adapt OctoPocus, a 2D gesture guiding system, to the case of 3D The OctoPocus3D guidance system displays a set of 3D gestures as 3D pipes and allows users to understand how the system processes gesture input Several feedback and feedforward visual alternatives are proposed in the literature However, their impact on guidance remains to be evaluated We report the results of two user experiments that aim at designing OctoPocus3D by exploring these alternatives The results show that a concurrent feedback, which visually simplifies the 3D scene during the execution of the gesture, increases the recognition rate, but only during the first two repetitions After the first two repetitions, users achieve the same recognition rate with a terminal feedback (after the execution of the gesture), a concurrent feedback, both or neither With respect to feedforward, the overall stability of the 3D scene explored through the origin of the pipes during the execution of the gestures does not influence the recognition rate or the execution time Finally, the results also show that displaying upcoming portions of the gestures allows 8% faster completion times than displaying the complete remaining portions This indicates that preventing visual clutter of the 3D scene prevails over gesture anticipation

/pdf/designing-3d-gesture-guidance-visual-feedback-and-335yaucvd3.pdf

Designing 3D Gesture Guidance: Visual Feedback and Feedforward Design Options

Smartwatches enable rapid access to information anytime and anywhere. However, current smartwatch content navigation techniques, for panning and zooming, were directly adopted from those used on smartphones. These techniques are cumbersome when performed on small smartwatch screens and have not been evaluated for their support in mobility and encumbrance contexts (when the user's hands are busy). We studied the effect of mobility and encumbrance on common content navigation techniques and found a significant decrease in performance as the pace of mobility increases or when the user was encumbered with busy hands. Based on these initial findings, we proposed a design space which would improve efficiency when navigation techniques, such as panning and zooming, are employed in mobility contexts. Our results reveal that our design space can effectively be used to create novel interaction techniques that improve smartwatch content navigation in mobility and encumbrance contexts.

/pdf/d-swime-a-design-space-for-smartwatch-interaction-techniques-41hn0tc6jw.pdf

D-SWIME: A Design Space for Smartwatch Interaction Techniques Supporting Mobility and Encumbrance

Users can benefit from using an auxiliary peripheral that could mitigate many concerns with direct text entry on wearable devices. We introduce ThumbText, a thumb-operated text entry approach for a ring-sized touch surface. Through a multi-part exploration, we first identify a suitable discretization of the miniature touch surface for thumb input. We then design a number of two-step selection techniques for supporting the input of at least 28 characters. On a miniature touch surface, we find that a continuous touch-slide-lift selection technique in a 2×3 grid discretization offers improved performance gains over other selection methods. Finally, we evaluate ThumbText against techniques also designed for wearable devices and find that ThumbText allows for higher text entry rates than SwipeBoard and H4-Writer.

ThumbText: Text Entry for Wearable Devices Using a Miniature Ring

We propose the concept of a guiding system specifically designed for semaphoric gaze gestures, i.e. gestures defining a vocabulary to trigger commands via the gaze modality. Our design exploration considers fundamental gaze gesture phases: Exploration, Guidance, and Return. A first experiment reveals that Guidance with dynamic elements moving along 2D paths is efficient and resistant to visual complexity. A second experiment reveals that a Rapid Serial Visual Presentation of command names during Exploration allows for more than 30% faster command retrievals than a standard visual search. To resume the task where the guide was triggered, labels moving from the outward extremity of 2D paths toward the guide center leads to efficient and accurate origin retrieval during the Return phase. We evaluate our resulting Gaze Gesture Guiding system, G3, for interacting with distant objects in an office environment using a head-mounted display. Users report positively on their experience with both semaphoric gaze gestures and G3.

/pdf/designing-a-gaze-gesture-guiding-system-2zs2aku68r.pdf

William Delamare

Papers

Designing guiding systems for gesture-based interaction

Designing 3D Gesture Guidance: Visual Feedback and Feedforward Design Options

D-SWIME: A Design Space for Smartwatch Interaction Techniques Supporting Mobility and Encumbrance

ThumbText: Text Entry for Wearable Devices Using a Miniature Ring

Designing a gaze gesture guiding system