Augmented Reality (AR) interfaces have been studied extensively over the last few decades, with a growing number of user-based experiments. In this paper, we systematically review 10 years of the most influential AR user studies, from 2005 to 2014. A total of 291 papers with 369 individual user studies have been reviewed and classified based on their application areas. The primary contribution of the review is to present the broad landscape of user-based AR research, and to provide a high-level view of how that landscape has changed. We summarize the high-level contributions from each category of papers, and present examples of the most influential user studies. We also identify areas where there have been few user studies, and opportunities for future research. Among other things, we find that there is a growing trend toward handheld AR user studies, and that most studies are conducted in laboratory settings and do not involve pilot testing. This research will be useful for AR researchers who want to follow best practices in designing their own AR user studies.

/pdf/a-systematic-review-of-10-years-of-augmented-reality-1zfm5ljlgk.pdf

A Systematic Review of 10 Years of Augmented Reality Usability Studies: 2005 to 2014

In this paper, a review is presented for the research on eye gaze estimation techniques and applications, which has progressed in diverse ways over the past two decades. Several generic eye gaze use-cases are identified: desktop, TV, head-mounted, automotive, and handheld devices. Analysis of the literature leads to the identification of several platform specific factors that influence gaze tracking accuracy. A key outcome from this review is the realization of a need to develop standardized methodologies for the performance evaluation of gaze tracking systems and achieve consistency in their specification and comparative evaluation. To address this need, the concept of a methodological framework for practical evaluation of different gaze tracking systems is proposed.

A Review and Analysis of Eye-Gaze Estimation Systems, Algorithms and Performance Evaluation Methods in Consumer Platforms

A novel user interface (UI) design based on the personality characteristics of users was proposed and examined in a mobile learning context. It was argued that differences in personality can stimulate individuals’ information processing capabilities in according to their display preferences, thus an effective visual experience. The personality characteristics and design preferences of 87 students (37 male, and 50 female) were collected and analysed. The clustering result (using k-means algorithm) revealed two potential personality types, which we call the neuroticism and the extra-conscientiousness groups. Then, an interface was designed for each personality group using the association rules method. An eye-tracking device was used to record changes in participants’ eye-pupil diameter and fixation duration, and thus examine their cognitive load and attention. The participants’ eye movement data of each group showed that their visual experience was significantly improved when using the interface designed based on their personality characteristics. This work offers some important design and practical insights to the human–computer interaction and the design of mobile device UI.

Towards incorporating personality into the design of an interface: a method for facilitating users’ interaction with the display

Human eye gaze has recently been used as an effective input interface for wearable displays. In this paper, we propose a gaze-based interaction framework for optical see-through displays. The proposed system can automatically judge whether a user is engaged with virtual content in the display or focused on the real environment and can determine his or her cognitive state. With these analytic capacities, we implement several proactive system functions including adaptive brightness, scrolling, messaging, notification, and highlighting, which would otherwise require manual interaction. The goal is to manage the relationship between virtual and real, creating a more cohesive and seamless experience for the user. We conduct user experiments including attention engagement and cognitive state analysis, such as reading detection and gaze position estimation in a wearable display towards the design of augmented reality text display applications. The results from the experiments show robustness of the attention engagement and cognitive state analysis methods. A majority of the experiment participants (8/12) stated the proactive system functions are beneficial.

Attention Engagement and Cognitive State Analysis for Augmented Reality Text Display Functions

The subject matter described herein relates to methods, systems, and computer readable media for leveraging user gaze in a user monitoring subregion selection system. One system includes a first camera configured to capture an image of a scene. In one implementation, the system includes a pattern generator configured to generate and project a pattern onto a surface of at least one of a user's eyes. The system further includes a sensor configured to obtain an image of the pattern reflected from at least one of the user's eyes. In an alternate implementation, the pattern generator may be omitted, and the sensor may be a stereo user facing camera. The system further includes a gaze estimation and scene mapping module configured to estimate a gaze direction of the user using the image captured by the user facing sensor and to map the estimated gaze direction to the image of the scene based on a location of an object of interest within the scene. The system further includes a subregion selection module configured to select a subregion of the image of the scene based on the mapped user gaze direction.

Methods, systems, and computer readable media for leveraging user gaze in user monitoring subregion selection systems

We present a new system that assists people's reading activity by combining a wearable eye tracker, a see-through head mounted display, and an image based document retrieval engine. An image based document retrieval engine is used for identification of the reading document, whereas an eye tracker is used to detect which part of the document the reader is currently reading. The reader can refer to the glossary of the latest viewed key word by looking at the see-through head mounted display. This novel document reading assist application, which is the integration of a document retrieval system into an everyday reading scenario for the first time, enriches people's reading life. In this paper, we i) investigate the performance of the state-of-the-art image based document retrieval method using a wearable camera, ii) propose a method for identification of the word the reader is attendant, and iii) conduct pilot studies for evaluation of the system in this reading context. The results show the potential of a document retrieval system in combination with a gaze based user-oriented system.

/pdf/wearable-reading-assist-system-augmented-reality-document-x50ppo24hk.pdf

Wearable Reading Assist System: Augmented Reality Document Combining Document Retrieval and Eye Tracking

When we read a document (any kind of, scientific papers, novels, etc.), we often encounter a situation that the information from the reading document is too less to comprehend what the author(s) would like to convey. In this paper, we demonstrate how the combination of a wearable eye tracker, a see-through head-mounted display (HMD) and an image based document retrieval engine enhances people's reading experiences. By using our proposed system, the reader can get supportive information in the see-through HMD when he wants. A wearable eye tracker and a document retrieval engine are used to detect which line in the document the reader is reading. We propose a method to detect the reader's attention on a word in a reading document, in order to present information at a preferable moment. Furthermore, we also propose a method to project a point of the document to a point of the HMD screen, by calculating the pose of the reading document in the camera image. This projection enables the system to overlay the information dynamically in an augmented view on the reading line. The results from the user study and the experiments show the potential of the proposed system in a practical use case.

Wakana Suzuki

Papers

Wearable Reading Assist System: Augmented Reality Document Combining Document Retrieval and Eye Tracking

User attention oriented augmented reality on documents with document dependent dynamic overlay