The success of deep learning has been a catalyst to solving increasingly complex machine-learning problems, which often involve multiple data modalities. We review recent advances in deep multimodal learning and highlight the state-of the art, as well as gaps and challenges in this active research field. We first classify deep multimodal learning architectures and then discuss methods to fuse learned multimodal representations in deep-learning architectures. We highlight two areas of research&#x02013;regularization strategies and methods that learn or optimize multimodal fusion structures&#x02013;as exciting areas for future work.

Deep Multimodal Learning: A Survey on Recent Advances and Trends

We introduce hand movement, orientation, and grasp (HMOG), a set of behavioral features to continuously authenticate smartphone users. HMOG features unobtrusively capture subtle micro-movement and orientation dynamics resulting from how a user grasps, holds, and taps on the smartphone. We evaluated authentication and biometric key generation (BKG) performance of HMOG features on data collected from 100 subjects typing on a virtual keyboard. Data were collected under two conditions: 1) sitting and 2) walking. We achieved authentication equal error rates (EERs) as low as 7.16% (walking) and 10.05% (sitting) when we combined HMOG, tap, and keystroke features. We performed experiments to investigate why HMOG features perform well during walking. Our results suggest that this is due to the ability of HMOG features to capture distinctive body movements caused by walking, in addition to the hand-movement dynamics from taps. With BKG, we achieved the EERs of 15.1% using HMOG combined with taps. In comparison, BKG using tap, key hold, and swipe features had EERs between 25.7% and 34.2%. We also analyzed the energy consumption of HMOG feature extraction and computation. Our analysis shows that HMOG features extracted at a 16-Hz sensor sampling rate incurred a minor overhead of 7.9% without sacrificing authentication accuracy. Two points distinguish our work from current literature: 1) we present the results of a comprehensive evaluation of three types of features (HMOG, keystroke, and tap) and their combinations under the same experimental conditions and 2) we analyze the features from three perspectives (authentication, BKG, and energy consumption on smartphones).

/pdf/hmog-new-behavioral-biometric-features-for-continuous-wxqxi7t4mv.pdf

HMOG: New Behavioral Biometric Features for Continuous Authentication of Smartphone Users

Recent developments in sensing and communication technologies have led to an explosion in the use of mobile devices such as smartphones and tablets. With the increase in the use of mobile devices, users must constantly worry about security and privacy, as the loss of a mobile device could compromise personal information. To deal with this problem, continuous authentication systems (also known as active authentication systems) have been proposed, in which users are continuously monitored after initial access to the mobile device. In this article, we provide an overview of different continuous authentication methods on mobile devices. We discuss the merits and drawbacks of the available approaches and identify promising avenues of research in this rapidly evolving field.

Continuous User Authentication on Mobile Devices: Recent progress and remaining challenges

Security and privacy challenges in mobile cloud computing

State-of-the-art password guessing tools, such as HashCat and John the Ripper, enable users to check billions of passwords per second against password hashes. In addition to performing straightforward dictionary attacks, these tools can expand password dictionaries using password generation rules, such as concatenation of words (e.g., “password123456”) and leet speak (e.g., “password” becomes “p4s5w0rd”). Although these rules work well in practice, creating and expanding them to model further passwords is a labor-intensive task that requires specialized expertise.

PassGAN: A Deep Learning Approach for Password Guessing

In this paper, we introduce Hand Movement, Orientation, and Grasp (HMOG), a behavioral biometric to continuously authenticate smartphone users. HMOG unobtrusively captures subtle micro-movement and orientation dynamics resulting from how a user grasps, holds, and taps on the smartphone. We evaluated the authentication and biometric key generation (BKG) performance of HMOG features on typing data collected from 100 subjects. Data was collected under two conditions: sitting and walking. We achieved authentication EERs as low as 6.92% (walking) and 10.20% (sitting) when we combined HMOG, tap, and keystroke features. We performed additional experiments to investigate why HMOG features perform well during walking. Our results suggest that this is due to the ability of HMOG features to capture distinctive body movements caused by walking, in addition to the hand-movement dynamics from taps. With BKG, we achieved EERs of 13.7% using HMOG combined with taps. In comparison, BKG using tap, key hold, and swipe features led to EERs between 31.3% and 36.7%. We also analyzed the energy consumption of HMOG feature extraction and computation. Our analysis shows that HMOG features extracted at 16Hz sensor sampling rate incurred a minor overhead of 7.9% without sacrificing authentication accuracy.

HMOG: A New Biometric Modality for Continuous Authentication of Smartphone Users.

Most continuous user authentication techniques based on typing behavior rely on the keystroke dynamics or on the linguistic style of the user However, there is a rich spectrum of cognition-centric behavioral traits that a typist exhibits during different stages of text production (eg, composition, translation, and revision), which to our knowledge, have not been considered for continuous authentication We study the continuous authentication performance of 123 behavioral traits extracted from discrete cognitive units called bursts We performed experiments on typing data collected from 486 volunteer subjects Our findings include: (1) features from bursts delimited by pause events have significantly higher availability and authentication performance compared to bursts delimited by revision events; (2) bursts with pause durations of at least one second provide the best authentication accuracy and availability; and (3) fusing our features with traditional keystroke dynamics features reduced authentication error rates We achieved an equal error rate between 1337 and 455 percent for authentication windows as low as 30 seconds to 35 minutes

Continuous authentication with cognition-centric text production and revision features

We introduce Hand Movement, Orientation, and Grasp (HMOG), a set of behavioral features to continuously authenticate smartphone users. HMOG features unobtrusively capture subtle micro-movement and orientation dynamics resulting from how a user grasps, holds, and taps on the smartphone. We evaluated authentication and biometric key generation (BKG) performance of HMOG features on data collected from 100 subjects typing on a virtual keyboard. Data was collected under two conditions: sitting and walking. We achieved authentication EERs as low as 7.16% (walking) and 10.05% (sitting) when we combined HMOG, tap, and keystroke features. We performed experiments to investigate why HMOG features perform well during walking. Our results suggest that this is due to the ability of HMOG features to capture distinctive body movements caused by walking, in addition to the hand-movement dynamics from taps. With BKG, we achieved EERs of 15.1% using HMOG combined with taps. In comparison, BKG using tap, key hold, and swipe features had EERs between 25.7% and 34.2%. We also analyzed the energy consumption of HMOG feature extraction and computation. Our analysis shows that HMOG features extracted at 16Hz sensor sampling rate incurred a minor overhead of 7.9% without sacrificing authentication accuracy. Two points distinguish our work from current literature: 1) we present the results of a comprehensive evaluation of three types of features (HMOG, keystroke, and tap) and their combinations under the same experimental conditions, and 2) we analyze the features from three perspectives (authentication, BKG, and energy consumption on smartphones).

Zdenka Sitova

Papers

HMOG: New Behavioral Biometric Features for Continuous Authentication of Smartphone Users

HMOG: A New Biometric Modality for Continuous Authentication of Smartphone Users.

Continuous authentication with cognition-centric text production and revision features

HMOG: New Behavioral Biometric Features for Continuous Authentication of Smartphone Users