Yi Xu

Bio: Yi Xu is an academic researcher from University of North Carolina at Chapel Hill. The author has contributed to research in topics: CAPTCHA & Usability. The author has an hindex of 6, co-authored 9 publications receiving 278 citations. Previous affiliations of Yi Xu include Tsinghua University.

Seeing double: reconstructing obscured typed input from repeated compromising reflections

Abstract: Of late, threats enabled by the ubiquitous use of mobile devices have drawn much interest from the research community. However, prior threats all suffer from a similar, and profound, weakness - namely the requirement that the adversary is either within visual range of the victim (e.g., to ensure that the pop-out events in reflections in the victim's sunglasses can be discerned) or is close enough to the target to avoid the use of expensive telescopes. In this paper, we broaden the scope of the attacks by relaxing these requirements and show that breaches of privacy are possible even when the adversary is around a corner. The approach we take overcomes challenges posed by low image resolution by extending computer vision methods to operate on small, high-noise, images. Moreover, our work is applicable to all types of keyboards because of a novel application of fingertip motion analysis for key-press detection. In doing so, we are also able to exploit reflections in the eyeball of the user or even repeated reflections (i.e., a reflection of a reflection of the mobile device in the eyeball of the user). Our empirical results show that we can perform these attacks with high accuracy, and can do so in scenarios that aptly demonstrate the realism of this threat.

Virtual U: defeating face liveness detection by building virtual models from your public photos

Abstract: In this paper, we introduce a novel approach to bypass modern face authentication systems. More specifically, by leveraging a handful of pictures of the target user taken from social media, we show how to create realistic, textured, 3D facial models that undermine the security of widely used face authentication solutions. Our framework makes use of virtual reality (VR) systems, incorporating along the way the ability to perform animations (e.g., raising an eyebrow or smiling) of the facial model, in order to trick liveness detectors into believing that the 3D model is a real human face. The synthetic face of the user is displayed on the screen of the VR device, and as the device rotates and translates in the real world, the 3D face moves accordingly. To an observing face authentication system, the depth and motion cues of the display match what would be expected for a human face. We argue that such VR-based spoofing attacks constitute a fundamentally new class of attacks that point to a serious weaknesses in camera-based authentication systems: Unless they incorporate other sources of verifiable data, systems relying on color image data and camera motion are prone to attacks via virtual realism. To demonstrate the practical nature of this threat, we conduct thorough experiments using an end-to-end implementation of our approach and show how it undermines the security of several face authentication solutions that include both motion-based and liveness detectors.

Security and usability challenges of moving-object CAPTCHAs: decoding codewords in motion

Abstract: We explore the robustness and usability of moving-image object recognition (video) captchas, designing and implementing automated attacks based on computer vision techniques. Our approach is suitable for broad classes of moving-image captchas involving rigid objects. We first present an attack that defeats instances of such a captcha (NuCaptcha) representing the state-of-the-art, involving dynamic text strings called codewords. We then consider design modifications to mitigate the attacks (e.g., overlapping characters more closely). We implement the modified captchas and test if designs modified for greater robustness maintain usability. Our lab-based studies show that the modified captchas fail to offer viable usability, even when the captcha strength is reduced below acceptable targets--signaling that the modified designs are not viable. We also implement and test another variant of moving text strings using the known emerging images idea. This variant is resilient to our attacks and also offers similar usability to commercially available approaches. We explain why fundamental elements of the emerging images concept resist our current attack where others fails.

Security Analysis and Related Usability of Motion-Based CAPTCHAs: Decoding Codewords in Motion

Abstract: We explore the robustness and usability of moving-image object recognition (video) captchas, designing and implementing automated attacks based on computer vision techniques. Our approach is suitable for broad classes of moving-image captchas involving rigid objects. We first present an attack that defeats instances of such a captcha (NuCaptcha) representing the state-of-the-art, involving dynamic text strings called codewords. We then consider design modifications to mitigate the attacks (e.g., overlapping characters more closely, randomly changing the font of individual characters, or even randomly varying the number of characters in the codeword). We implement the modified captchas and test if designs modified for greater robustness maintain usability. Our lab-based studies show that the modified captchas fail to offer viable usability, even when the captcha strength is reduced below acceptable targets. Worse yet, our GPU-based implementation shows that our automated approach can decode these captchas faster than humans can, and we can do so at a relatively low cost of roughly 50 cents per 1,000 captchas solved based on Amazon EC2 rates circa 2012. To further demonstrate the challenges in designing usable captchas, we also implement and test another variant of moving text strings using the known emerging images concept. This variant is resilient to our attacks and also offers similar usability to commercially available approaches. We explain why fundamental elements of the emerging images idea resist our current attack where others fail.

Watching the Watchers: Automatically Inferring TV Content From Outdoor Light Effusions

Abstract: The flickering lights of content playing on TV screens in our living rooms are an all too familiar sight at night --- and one that many of us have paid little attention to with regards to the amount of information these diffusions may leak to an inquisitive outsider. In this paper, we introduce an attack that exploits the emanations of changes in light (e.g., as seen through the windows and recorded over 70 meters away) to reveal the programs we watch. Our empirical results show that the attack is surprisingly robust to a variety of noise signals that occur in real-world situations, and moreover, can successfully identify the content being watched among a reference library of tens of thousands of videos within several seconds. The robustness and efficiency of the attack can be attributed to the use of novel feature sets and an elegant online algorithm for performing index-based matches.

HMOG: New Behavioral Biometric Features for Continuous Authentication of Smartphone Users

Abstract: We introduce hand movement, orientation, and grasp (HMOG), a set of behavioral features to continuously authenticate smartphone users. HMOG features unobtrusively capture subtle micro-movement and orientation dynamics resulting from how a user grasps, holds, and taps on the smartphone. We evaluated authentication and biometric key generation (BKG) performance of HMOG features on data collected from 100 subjects typing on a virtual keyboard. Data were collected under two conditions: 1) sitting and 2) walking. We achieved authentication equal error rates (EERs) as low as 7.16% (walking) and 10.05% (sitting) when we combined HMOG, tap, and keystroke features. We performed experiments to investigate why HMOG features perform well during walking. Our results suggest that this is due to the ability of HMOG features to capture distinctive body movements caused by walking, in addition to the hand-movement dynamics from taps. With BKG, we achieved the EERs of 15.1% using HMOG combined with taps. In comparison, BKG using tap, key hold, and swipe features had EERs between 25.7% and 34.2%. We also analyzed the energy consumption of HMOG feature extraction and computation. Our analysis shows that HMOG features extracted at a 16-Hz sensor sampling rate incurred a minor overhead of 7.9% without sacrificing authentication accuracy. Two points distinguish our work from current literature: 1) we present the results of a comprehensive evaluation of three types of features (HMOG, keystroke, and tap) and their combinations under the same experimental conditions and 2) we analyze the features from three perspectives (authentication, BKG, and energy consumption on smartphones).

Security and privacy for augmented reality systems

Abstract: AR systems pose potential security concerns that should be addressed before the systems become widespread.

When Good Becomes Evil: Keystroke Inference with Smartwatch

Abstract: One rising trend in today's consumer electronics is the wearable devices, e.g., smartwatches. With tens of millions of smartwatches shipped, however, the security implications of such devices are not fully understood. Although previous studies have pointed out some privacy concerns about the data that can be collected, like personalized health information, the threat is considered low as the leaked data is not highly sensitive and there is no real attack implemented. In this paper we investigate a security problem coming from sensors in smartwatches, especially the accelerometer. The results show that the actual threat is much beyond people's awareness. Being worn on the wrist, the accelerometer built within a smartwatch can track user's hand movements, which makes inferring user inputs on keyboards possible in theory. But several challenges need to be addressed ahead in the real-world settings: e.g., small and irregular hand movements occur persistently during typing, which degrades the tracking accuracy and sometimes even overwhelms useful signals. In this paper, we present a new and practical side-channel attack to infer user inputs on keyboards by exploiting sensors in smartwatch. Novel keystroke inference models are developed to mitigate the negative impacts of tracking noises. We focus on two major categories of keyboards: one is numeric keypad that is generally used to input digits, and the other is QWERTY keyboard on which a user can type English text. Two prototypes have been built to infer users' banking PINs and English text when they type on POS terminal and QWERTY keyboard respectively. Our results show that for numeric keyboard, the probability of finding banking PINs in the top 3 candidates can reach 65%, while for QWERTY keyboard, a significant accuracy improvement is achieved compared to the previous works, especially of the success rate of finding the correct word in the top 10 candidates.