The PASCAL Visual Object Classes Challenge

Despite the widespread use of mobile phones and tablets, hand-held portable devices have only recently been identified as a promising platform for gaze-aware applications. Estimating gaze on portable devices is challenging given their limited computational resources, low quality integrated front-facing RGB cameras, and small screens to which gaze is mapped. In this paper we present EyeTab, a model-based approach for binocular gaze estimation that runs entirely on an unmodified tablet. EyeTab builds on set of established image processing and computer vision algorithms and adapts them for robust and near-realtime gaze estimation. A technical prototype evaluation with eight participants in a normal indoors office setting shows that EyeTab achieves an average gaze estimation accuracy of 6.88° of visual angle at 12 frames per second.

EyeTab: model-based gaze estimation on unmodified tablet computers

In this thesis, we study the problem of face recognition based on image sets. The main objective of our work is to develop set-based distance metrics that are able to measure the similarity between image sets, rather than conventional distance metrics that can only measure the distance between samples. The face images obtained from real-life impose great challenges to the conventional face recognition systems. Large variations in appearances and various imperfections such as occlusions and misalignments in the face images severely degrade the recognition performance. One possible solution is to utilize more face images for each person, e.g., a collection of photos from personal galleries or frames extracted from a video clip. Under such circumstances, the face recognition task becomes the process of modelling and matching image sets. Our investigation then focuses on developing appropriate models and set-based distance metrics for representing different image sets.

Face recognition based on image sets

Conventional feature-based and model-based gaze estimation methods have proven to perform well in settings with controlled illumination and specialized cameras. In unconstrained real-world settings, however, such methods are surpassed by recent appearance-based methods due to difficulties in modeling factors such as illumination changes and other visual artifacts. We present a novel learning-based method for eye region landmark localization that enables conventional methods to be competitive to latest appearance-based methods. Despite having been trained exclusively on synthetic data, our method exceeds the state of the art for iris localization and eye shape registration on real-world imagery. We then use the detected landmarks as input to iterative model-fitting and lightweight learning-based gaze estimation methods. Our approach outperforms existing model-fitting and appearance-based methods in the context of person-independent and personalized gaze estimation.

Learning to find eye region landmarks for remote gaze estimation in unconstrained settings

We study gaze estimation on tablets; our key design goal is uncalibrated gaze estimation using the front-facing camera during natural use of tablets, where the posture and method of holding the tablet are not constrained. We collected a large unconstrained gaze dataset of tablet users, labeled Rice TabletGaze dataset. The dataset consists of 51 subjects, each with 4 different postures and 35 gaze locations. Subjects vary in race, gender and in their need for prescription glasses, all of which might impact gaze estimation accuracy. We made three major observations on the collected data and employed a baseline algorithm for analyzing the impact of several factors on gaze estimation accuracy. The baseline algorithm is based on multilevel HoG feature and Random Forests regressor, which achieves a mean error of 3.17 cm. We perform extensive evaluation on the impact of various practical factors such as person dependency, dataset size, race, wearing glasses and user posture on the gaze estimation accuracy.

TabletGaze: dataset and analysis for unconstrained appearance-based gaze estimation in mobile tablets

Reading is a ubiquitous activity that many people even perform in transit, such as while on the bus or while walking. Tracking reading enables us to gain more insights about expertise level and potential knowledge of users -- towards a reading log tracking and improve knowledge acquisition. As a first step towards this vision, in this work we investigate whether different document types can be automatically detected from visual behaviour recorded using a mobile eye tracker. We present an initial recognition approach that com- bines special purpose eye movement features as well as machine learning for document type detection. We evaluate our approach in a user study with eight participants and five Japanese document types and achieve a recognition performance of 74% using user-independent training.

/pdf/i-know-what-you-are-reading-recognition-of-document-types-2jtqfozgkt.pdf

I know what you are reading: recognition of document types using mobile eye tracking

As reading is an integral part of our knowledge lives, we should know more about our reading activities. This paper introduces a reading application for smart phone and tablets that aims at giving user more quantified information about their reading habits. We present our work towards building an open library for eye tracking on unmodified tablets and smart phones to support some of the applications advanced functionality. We implemented already several eye tracking algorithms from previous work, unfortunately all seem not to be robust enough for our application case. We give an overview about our challenges and potential solutions.

My reading life: towards utilizing eyetracking on unmodified tablets and phones

Recognition of user activities is a key issue for context-aware computing. We present a method for recognition of user daily activities using gaze motion features and image-based visual features. Gaze motion features dominate for inferring the user's egocentric context whereas image-based visual features dominate for recognition of the environments and the target objects. The experimental results show the fusion of those different type of features improves performance of user daily activity recognition.

Daily activity recognition combining gaze motion and visual features

Have you ever experienced that you cannot remember the name of a person you meet again? To circumvent such an awkward situation, it would be great if you had had a system that tells you the name of the person in secret In this paper, we propose a wearable system of real-time face recognition to support human memory The contributions of our work are summarized as follows: (1) We discuss the design and implementation details of a wearable system capable of augmenting human memory by vision-based realtime face recognition (2) We propose a 2 step recognition approach from coarse-to-fine grain to boost the execution time towards the social acceptable limit of 900 [ms] (3) In experiments, we evaluate the computational time and recognition rate As results, the proposed system could recognize a face in 238 ms with the the cumulative recognition rate at the 10th rank was 933 % Computational time with the coarse-to-fine search was 668 ms less than that without coarse-to-fine search and the results showed that the proposed system has enough ability to recognize faces in real time

/pdf/who-are-you-a-wearable-face-recognition-system-to-support-2fjg3gk4b7.pdf

Who are you?: A wearable face recognition system to support human memory

Most gait recognition approaches rely on silhouette-based representations due to high recognition accu- racy and computational efficiency, and a key problem for those approaches is how to accurately extract individuality- preserved silhouettes from real scenes, where foreground colors may be similar to background colors and the back- groundis cluttered. We thereforeproposea method of individuality-preservingsilhouetteextractionfor gait recognition using standard gait models (SGMs) composed of clean silhouette sequences of a variety of training subjects as a shape prior. We firstly match the multiple SGMs to a background subtraction sequence of a test subject by dynamic pro- gramming and select the training subject whose SGM fit the test sequence the best. We then formulate our silhouette extraction problem in a well-established graph-cut segmentation framework while considering a balance between the observed test sequence and the matched SGM. More specifically, we define an energy function to be minimized by the following three terms: (1) a data term derived from the observed test sequence, (2) a smoothness term derived from spatio-temporally adjacent edges, and (3) a shape-prior term derived from the matched SGM. We demonstrate that the proposed method successfully extracts individuality-preserved silhouettes and improved gait recognition accuracy through experiments using 56 subjects.

/pdf/individuality-preserving-silhouette-extraction-for-gait-2tttexmyyi.pdf

Yuzuko Utsumi

Papers

I know what you are reading: recognition of document types using mobile eye tracking

My reading life: towards utilizing eyetracking on unmodified tablets and phones

Daily activity recognition combining gaze motion and visual features

Who are you?: A wearable face recognition system to support human memory

Individuality-preserving Silhouette Extraction for Gait Recognition