Despite active research and significant progress in the last 30 years, eye detection and tracking remains challenging due to the individuality of eyes, occlusion, variability in scale, location, and light conditions. Data on eye location and details of eye movements have numerous applications and are essential in face detection, biometric identification, and particular human-computer interaction tasks. This paper reviews current progress and state of the art in video-based eye detection and tracking in order to identify promising techniques as well as issues to be further addressed. We present a detailed review of recent eye models and techniques for eye detection and tracking. We also survey methods for gaze estimation and compare them based on their geometric properties and reported accuracies. This review shows that, despite their apparent simplicity, the development of a general eye detection technique involves addressing many challenges, requires further theoretical developments, and is consequently of interest to many other domains problems in computer vision and beyond.

/pdf/in-the-eye-of-the-beholder-a-survey-of-models-for-eyes-and-oxe7gfw5jq.pdf

In the Eye of the Beholder: A Survey of Models for Eyes and Gaze

This paper describes a real-time online prototype driver-fatigue monitor. It uses remotely located charge-coupled-device cameras equipped with active infrared illuminators to acquire video images of the driver. Various visual cues that typically characterize the level of alertness of a person are extracted in real time and systematically combined to infer the fatigue level of the driver. The visual cues employed characterize eyelid movement, gaze movement, head movement, and facial expression. A probabilistic model is developed to model human fatigue and to predict fatigue based on the visual cues obtained. The simultaneous use of multiple visual cues and their systematic combination yields a much more robust and accurate fatigue characterization than using a single visual cue. This system was validated under real-life fatigue conditions with human subjects of different ethnic backgrounds, genders, and ages; with/without glasses; and under different illumination conditions. It was found to be reasonably robust, reliable, and accurate in fatigue characterization.

/pdf/real-time-nonintrusive-monitoring-and-prediction-of-driver-4x0a6stl0p.pdf

Real-time nonintrusive monitoring and prediction of driver fatigue

Eye gaze tracking techniques for interactive applications

This work explores a new direction in utilizing eye gaze for computer input. Gaze tracking has long been considered as an alternative or potentially superior pointing method for computer input. We believe that many fundamental limitations exist with traditional gaze pointing. In particular, it is unnatural to overload a perceptual channel such as vision with a motor control task. We therefore propose an alternative approach, dubbed MAGIC (Manual And Gaze Input Cascaded) pointing. With such an approach, pointing appears to the user to be a manual task, used for fine manipulation and selection. However, a large portion of the cursor movement is eliminated by warping the cursor to the eye gaze area, which encompasses the target. Two specific MAGIC pointing techniques, one conservative and one liberal, were designed, analyzed, and implemented with an eye tracker we developed. They were then tested in a pilot study. This early- stage exploration showed that the MAGIC pointing techniques might offer many advantages, including reduced physical effort and fatigue as compared to traditional manual pointing, greater accuracy and naturalness than traditional gaze pointing, and possibly faster speed than manual pointing. The pros and cons of the two techniques are discussed in light of both performance data and subjective reports.

/pdf/manual-and-gaze-input-cascaded-magic-pointing-2hny9xx29s.pdf

Manual and gaze input cascaded (MAGIC) pointing

This paper presents a general theory for the remote estimation of the point-of-gaze (POG) from the coordinates of the centers of the pupil and corneal reflections. Corneal reflections are produced by light sources that illuminate the eye and the centers of the pupil and corneal reflections are estimated in video images from one or more cameras. The general theory covers the full range of possible system configurations. Using one camera and one light source, the POG can be estimated only if the head is completely stationary. Using one camera and multiple light sources, the POG can be estimated with free head movements, following the completion of a multiple-point calibration procedure. When multiple cameras and multiple light sources are used, the POG can be estimated following a simple one-point calibration procedure. Experimental and simulation results suggest that the main sources of gaze estimation errors are the discrepancy between the shape of real corneas and the spherical corneal shape assumed in the general theory, and the noise in the estimation of the centers of the pupil and corneal reflections. A detailed example of a system that uses the general theory to estimate the POG on a computer screen is presented.

General theory of remote gaze estimation using the pupil center and corneal reflections

Recently, some video-based eye-gaze detection methods used in eye-slaved support systems for the severely disabled have been studied. In these methods, infrared light was irradiated to an eye, two feature areas (the corneal reflection light and pupil) were detected in the image obtained from a video camera and then the eye-gaze direction was determined by the relative positions between the two. However, there were problems concerning stable pupil detection under various room light conditions. In this paper, methods for precisely detecting the two feature areas are consistently mentioned. First, a pupil detection technique using two light sources and the image difference method is proposed. Second, for users wearing eye glasses, a method for eliminating the images of the light sources reflected in the glass lens is proposed. The effectiveness of these proposed methods is demonstrated by using an imaging board. Finally, the feasibility of implementing hardware for the proposed methods in real time is discussed.

Improved video-based eye-gaze detection method

Effectiveness of pupil area detection technique using two light sources and image difference method

Video-based eye-gaze point detection methods used in the support systems for the disabled have been studied. Recently, we proposed the pupil detection technique using two light sources and the image difference method, and presented its effectiveness. In this paper, a method eliminating the light sources reflected in eye glass lenses and precisely detecting the pupil and the glint (the corneal reflection light), which are the feature points for the determination of the eye-gaze point, were proposed. >

A gaze point detection device 1 comprises two stereo cameras 2 a, 2 b for acquiring a face image of a subject A, light sources 3 a, 3 b disposed on the outside of apertures 9 a, 9 b, a control circuit 4, 5, 6, and an image processor 7. The image processor 7 calculates a vector r from a corneal reflection point to a pupil on a plane perpendicular to reference lines of the stereo cameras 2 a, 2 b, computes an angle θ of lines of sight of the subject A with respect to the respective reference lines by using a function f1, corrects the function f1 such that directions of lines of sight are closer to each other, and calculates the line of sight directions to detect a gaze point Q on a display screen.

Yoshinobu Ebisawa

Papers

Improved video-based eye-gaze detection method

Effectiveness of pupil area detection technique using two light sources and image difference method

Improved video-based eye-gaze detection method

Gaze point detection method and gaze point detection device

Unconstrained pupil detection technique using two light sources and the image difference method.