Microsoft Kinect, a low-cost motion sensing device, enables users to interact with computers or game consoles naturally through gestures and spoken commands without any other peripheral equipment. As such, it has commanded intense interests in research and development on the Kinect technology. In this paper, we present, a comprehensive survey on Kinect applications, and the latest research and development on motion recognition using data captured by the Kinect sensor. On the applications front, we review the applications of the Kinect technology in a variety of areas, including healthcare, education and performing arts, robotics, sign language recognition, retail services, workplace safety training, as well as 3D reconstructions. On the technology front, we provide an overview of the main features of both versions of the Kinect sensor together with the depth sensing technologies used, and review literatures on human motion recognition techniques used in Kinect applications. We provide a classification of motion recognition techniques to highlight the different approaches used in human motion recognition. Furthermore, we compile a list of publicly available Kinect datasets. These datasets are valuable resources for researchers to investigate better methods for human motion recognition and lower-level computer vision tasks such as segmentation, object detection and human pose estimation.

/pdf/a-survey-of-applications-and-human-motion-recognition-with-hags1y076t.pdf

A Survey of Applications and Human Motion Recognition with Microsoft Kinect

This paper studies the security and memorability of free-form multitouch gestures for mobile authentication. Towards this end, we collected a dataset with a generate-test-retest paradigm where participants (N=63) generated free-form gestures, repeated them, and were later retested for memory. Half of the participants decided to generate one-finger gestures, and the other half generated multi-finger gestures. Although there has been recent work on template-based gestures, there are yet no metrics to analyze security of either template or free-form gestures. For example, entropy-based metrics used for text-based passwords are not suitable for capturing the security and memorability of free-form gestures. Hence, we modify a recently proposed metric for analyzing information capacity of continuous full-body movements for this purpose. Our metric computed estimated mutual information in repeated sets of gestures. Surprisingly, one-finger gestures had higher average mutual information. Gestures with many hard angles and turns had the highest mutual information. The best-remembered gestures included signatures and simple angular shapes. We also implemented a multitouch recognizer to evaluate the practicality of free-form gestures in a real authentication system and how they perform against shoulder surfing attacks. We discuss strategies for generating secure and memorable free-form gestures. We conclude that free-form gestures present a robust method for mobile authentication.

https://www.cs.helsinki.fi/u/ttonteri/pub/mobisys2014.pdf

Video: User-generated free-form gestures for authentication: security and memorability

The goal of this work is to enable user authentication via finger inputs on ubiquitous surfaces leveraging low-cost physical vibration. We propose VibWrite that extends finger-input authentication beyond touch screens to any solid surface for smart access systems (e.g., access to apartments, vehicles or smart appliances). It integrates passcode, behavioral and physiological characteristics, and surface dependency together to provide a low-cost, tangible and enhanced security solution. VibWrite builds upon a touch sensing technique with vibration signals that can operate on surfaces constructed from a broad range of materials. It is significantly different from traditional password-based approaches, which only authenticate the password itself rather than the legitimate user, and the behavioral biometrics-based solutions, which usually involve specific or expensive hardware (e.g., touch screen or fingerprint reader), incurring privacy concerns and suffering from smudge attacks. VibWrite is based on new algorithms to discriminate fine-grained finger inputs and supports three independent passcode secrets including PIN number, lock pattern, and simple gestures by extracting unique features in the frequency domain to capture both behavioral and physiological characteristics such as contacting area, touching force, and etc. VibWrite is implemented using a single pair of low-cost vibration motor and receiver that can be easily attached to any surface (e.g., a door panel, a desk or an appliance). Our extensive experiments demonstrate that VibWrite can authenticate users with high accuracy (e.g., over 95% within two trials), low false positive rate (e.g., less 3%) and is robust to various types of attacks.

https://dl.acm.org/doi/pdf/10.1145/3133956.3133964

VibWrite: Towards Finger-input Authentication on Ubiquitous Surfaces via Physical Vibration

Mobile authentication is indispensable for preventing unauthorized access to multi-touch mobile devices. Existing mobile authentication techniques are often cumbersome to use and also vulnerable to shoulder-surfing and smudge attacks. This paper focuses on designing, implementing, and evaluating TouchIn, a two-factor authentication system on multi-touch mobile devices. TouchIn works by letting a user draw on the touchscreen with one or multiple fingers to unlock his mobile device, and the user is authenticated based on the geometric properties of his drawn curves as well as his behavioral and physiological characteristics. TouchIn allows the user to draw on arbitrary regions on the touchscreen without looking at it. This nice sightless feature makes TouchIn very easy to use and also robust to shoulder-surfing and smudge attacks. Comprehensive experiments on Android devices confirm the high security and usability of TouchIn.

TouchIn: Sightless Two-factor Authentication on Multi-touch Mobile Devices

Draw-a-pin

Password-based authentication is easy to use but its security is bounded by how much a user can remember. Biometrics-based authentication requires no memorization but ‘resetting’ a biometric password may not always be possible. In this paper, we propose a user-friendly authentication system (KinWrite) that allows users to choose arbitrary, short and easy-to-memorize passwords while providing resilience to password cracking and password theft. KinWrite lets users write their passwords in 3D space and captures the handwriting motion using a low cost motion input sensing device—Kinect. The low resolution and noisy data captured by Kinect, combined with low consistency of in-space handwriting, have made it challenging to verify users. To overcome these challenges, we exploit the Dynamic Time Warping (DTW) algorithm to quantify similarities between handwritten passwords. Our experimental results involving 35 signatures from 18 subjects and a brute-force attacker have shown that KinWrite can achieve a 100% precision and a 70% recall (the worst case) for verifying honest users, encouraging us to carry out a much larger scale study towards designing a foolproof system.

/pdf/kinwrite-handwriting-based-authentication-using-kinect-4rg2697sja.pdf

KinWrite: Handwriting-Based Authentication Using Kinect.

Challenge-response (CR) is an effective way to authenticate users even if the communication channel is insecure. Traditionally CR authentication relies on one-way hashes and shared secrets to verify the identities of users. Such a method cannot cope with an insider attack, where a user can obtained the secret (i.e., the response) from a legitimate user. To cope with it, we design a biometric-based CR authentication scheme (hereafter MoCRA 
), which is derived from the motions as a user operates emerging depth-sensor- based input devices, such as a Leap Motion controller. We envision that to authenticate a user, MoCRA randomly chooses a string (e.g., a few words), and the user has to write the string in the air. Using Leap Motion, MoCRA captures the user's writing movements and then extracts his / her handwriting style. After verifying that what the user writes matches what is asked for, MoCRA leverages a Support Vecter Machine (SVM) with co-occurrence matrices to model the handwriting styles and can reliably authenticate users, even if what they write is completely different every time. Evaluated on data from 24 subjects over 7 months, MoCRA managed to verify a user with an average of $1.18\%$ 1 . 18 % (Equal Error Rate) EER and to reject impostors with $2.45\%$ 2 . 45 % EER.

Challenge-Response Authentication Using In-Air Handwriting Style Verification

Handwriting in-space from Kinect depth and color information is a challenging task due to the high variability of signature characteristics for difierent individuals. In this paper, a user-friendly human computer interaction system is proposed and implemented based on Kinect handwriting. The flngertip is flrstly tracked by our detection method in every depth frame to generate 3D trajectory of handwriting, and then normalization and smoothing are performed before feature extraction. On this basis, the time sequence feature of 3D signature can be captured as an online character recognition method, and a joint recognition framework is proposed based on DTW and SVM for input vectors of difierent lengths. The evaluation on a handwriting in-space dataset of digits from 0 to 9 shows that the proposed recognition scheme can ofier a high recognition accuracy and a satisfying robustness to noisy data in digit recognition test even with small training number. Therefore, the method can be successfully applied in many Human Computer Interaction applications in real world.

http://www.joics.com/publishedpapers/2015_12_1_413_422.pdf

Online Kinect Handwritten Digit Recognition Based on Dynamic Time Warping and Support Vector Machine

Nowadays most IoT devices in smart homes rely on radio frequency channels for communication, making them exposed to various attacks. Existing methods using encryption keys may be inapplicable on these resource-constrained devices that cannot afford the computationally expensive encryption operations. Thus, in this paper we design a key-free communication method for such devices. In particular, we introduce the Home-limited Channel (HLC) that can be accessed only within a house yet inaccessible for an outside-house attacker. Utilizing HLCs, we propose a challenge-response mechanism to authenticate the communications inside a house. The advantages of the HlcAuth protocol are low cost, lightweight as well as key-free, and requiring no human intervention. We show that HlcAuth can defeat replay attacks, message-forgery attacks, and man-in-the-middle (MiTM) attacks, among others. HlcAuth achieves 100% true positive rate (TPR) within 4.2m for in-house devices while 0% false positive rate (FPR) for outside attackers.

Jing Tian

Papers

KinWrite: Handwriting-Based Authentication Using Kinect.

Challenge-Response Authentication Using In-Air Handwriting Style Verification

Online Kinect Handwritten Digit Recognition Based on Dynamic Time Warping and Support Vector Machine

HlcAuth: Key-free and Secure Communications via Home-Limited Channel

Network congestion avoidance strategy with particle filter