A well-known bottleneck of contemporary mobile devices is the inefficient and error-prone touchscreen keyboard. In this paper, we propose UbiK, an alternative portable text-entry method that allows user to make keystrokes on conventional surfaces, e.g., wood desktop. UbiK enables text-input experience similar to that on a physical keyboard, but it only requires a keyboard outline printed on the surface or a piece of paper atop. The core idea is to leverage the microphone on a mobile device to accurately localize the keystrokes. To achieve fine-grained, centimeter scale granularity, UbiK extracts and optimizes the location-dependent multipath fading features from the audio signals, and takes advantage of the dual-microphone interface to improve signal diversity. We implement UbiK as an Android application. Our experiments demonstrate that UbiK is able to achieve above 95% of localization accuracy. Field trial involving first-time users shows that UbiK can significantly improve text-entry speed over current on-screen keyboards.

/pdf/ubiquitous-keyboard-for-small-mobile-devices-harnessing-3dd2sxxk1e.pdf

Ubiquitous keyboard for small mobile devices: harnessing multipath fading for fine-grained keystroke localization

Authentication methods can be improved by considering implicit, individual behavioural cues. In particular, verifying users based on typing behaviour has been widely studied with physical keyboards. On mobile touchscreens, the same concepts have been applied with little adaptations so far. This paper presents the first reported study on mobile keystroke biometrics which compares touch-specific features between three different hand postures and evaluation schemes. Based on 20.160 password entries from a study with 28 participants over two weeks, we show that including spatial touch features reduces implicit authentication equal error rates (EER) by 26.4 - 36.8% relative to the previously used temporal features. We also show that authentication works better for some hand postures than others. To improve applicability and usability, we further quantify the influence of common evaluation assumptions: known attacker data, training and testing on data from a single typing session, and fixed hand postures. We show that these practices can lead to overly optimistic evaluations. In consequence, we describe evaluation recommendations, a probabilistic framework to handle unknown hand postures, and ideas for further improvements.

/pdf/improving-accuracy-applicability-and-usability-of-keystroke-1s6bpf4rth.pdf

Improving Accuracy, Applicability and Usability of Keystroke Biometrics on Mobile Touchscreen Devices

Eyes-free input is desirable for ubiquitous computing, since interacting with mobile and wearable devices often competes for visual attention with other devices and tasks. In this paper, we explore eyes-free typing on a touchpad using one thumb, wherein a user taps on an imaginary QWERTY keyboard while receiving text feedback on a separate screen. Our hypothesis is that users can transfer their typing ability obtained from visible keyboards to eyes-free use. We propose two statistical decoding algorithms to infer users’ eyes-free input: the absolute algorithm and the relative algorithm. The absolute algorithm infers user input based on the absolute position of touch endpoints, while the relative algorithm infers based on the vectors between successive touch endpoints. Evaluation results showed users could achieve satisfying performance with both algorithms. Text entry rate was 17-23 WPM (words per minute) depending on the algorithm used. In comparison, a baseline cursor-based text entry method yielded only 7.66 WPM. In conclusion, our research demonstrates for the first time the feasibility of thumb-based eyes-free typing, which provides a new possibility for text entry on ubiquitous computing platforms such as smart TVs and HMDs.

BlindType: Eyes-Free Text Entry on Handheld Touchpad by Leveraging Thumb's Muscle Memory

We present a data logging concept, tool, and analyses to facilitate studies of everyday mobile touch keyboard use and free typing behaviour: 1) We propose a filtering concept to log typing without recording readable text and assess reactions to filters with a survey (N=349). 2) We release an Android keyboard app and backend that implement this concept. 3) Based on a three-week field study (N=30), we present the first analyses of keyboard use and typing biometrics on such free text typing data in the wild, including speed, postures, apps, auto correction, and word suggestions. We conclude that research on mobile keyboards benefits from observing free typing beyond the lab and discuss ideas for further studies.

ResearchIME: A Mobile Keyboard Application for Studying Free Typing Behaviour in the Wild

It is our great pleasure to welcome you to the 2012 ACM International Conference on Human-Computer Interaction with Mobile Devices and Services -- MobileHCI 2012.

MobileHCI is the world's leading conference in the field of Human Computer Interaction concerned with portable and personal devices and with the services to which they enable access. Mobile HCI provides a multidisciplinary forum for academics, hardware and software developers, designers and practitioners to discuss the challenges and potential solutions for effective interaction with and through mobile devices, applications, and services.

The conference continues to attract a significant number of submissions; this year we received 212 valid paper submissions. We have continued our commitment to improve the quality of the review process. A senior program committee of 38 internationally renowned scientists from academia and industry was assembled. Each paper received 3 or more high-quality peer reviews, as well as an additional meta review by the assigned PC member. Following last year's successful cross-Atlantic, split committee meeting, the 38 committee members assembled in two locations (Palo Alto and Berlin) that were linked by audio and video connections. This provided an opportunity for the papers and reviews to be discussed in detail and all final decisions to be agreed upon by the Program Committee as a whole.

The outcome of this process was that 54 of the 212 submissions were accepted (25%) for inclusion in the final Program, to be presented in San Francisco in September 2012. Of these, 39 were full papers and 15 were notes. A shepherding process was also used in which 6 of the 54 accepted papers were revised and improved under the expert guidance of a dedicated committee member. In our commitment to continually improving the quality of the Program, 8 papers/notes were given special recognition of excellence by being nominated for consideration as a Best Paper. A jury, consisting of 5 members of the Program Committee, was established to judge which of these papers represented the highest caliber of research in the field to be deserving of the Best Paper award. The final decision is to be revealed at the conference itself.

Proceedings of the 14th international conference on Human-computer interaction with mobile devices and services

This Note introduces a keyboard design that affords ten-finger touch typing by utilizing a touch sensor on the back side of a device. Previous work has used physical buttons. Using a touch sensor has the benefit that it retains the form factor and does not insist on a peripheral device. Moreover, any layout can be used. However, it is difficult to hit targets on a flat surface with no haptic feedback. Sandwich Keyboard is a prototype that folds any three-row keyboard layout and thus, by retaining the finger-to-letter assignment, supports transfer. Sandwich Keyboard includes an algorithm for constant adaptation of key targets in the back. We also learned that the detection of key presses from finger release enhances the performance of touch-typing on a multitouch sensor. After eight hours of training, experienced typists of the QWERTY and of the Dvorak Standard Keyboard (DSK) layout reached 26.1 and 46.2 wpm, respectively. We discuss improvements necessary for further increasing both speed and accuracy.

Sandwich keyboard: fast ten-finger typing on a mobile device with adaptive touch sensing on the back side

Recent work has shown that a multitouch sensor attached to the back of a handheld device can allow rapid typing engaging all ten fingers. However, high error rates remain a problem, because the user can not see or feel key-targets on the back. We propose a machine learning approach that can significantly improve accuracy. The method considers hand anatomy and movement ranges of fingers. The key insight is a combination of keyboard and hand models in a hierarchical clustering method. This enables dynamic re-estimation of key-locations while typing to account for changes in hand postures and movement ranges of fingers. We also show that accuracy can be further improved with language models. Results from a user study show improvements of over 40% compared to the previously deployed "naive" approach. We examine entropy as a touch precision metric with respect to typing experience. We also find that the QWERTY layout is not ideal. Finally, we conclude with ideas for further improvements.

/pdf/improving-accuracy-in-back-of-device-multitouch-typing-a-1vqg1wu2g0.pdf

Oliver Schoenleben

Papers

Sandwich keyboard: fast ten-finger typing on a mobile device with adaptive touch sensing on the back side

Improving accuracy in back-of-device multitouch typing: a clustering-based approach to keyboard updating