Predictive perceptual compression for real time video communication

TLDR: This paper has developed an eye gaze-aware MPEG-2 transcoder that can perceptually re-encode a live video stream in real time and compensates the interim eye movements between the sampling and actual coding.

Abstract: Approximately 2 degrees in our 140 degree vision span has sharp vision. Many researchers have been fascinated by the idea of eye-tracking integrated perceptual compression of an image or video, yet any practical system has yet to emerge. The unique challenge presented by real time perceptual video streaming is how to handle the fast nature of the human eye and provide its integration with computationally intensive video transcoding scheme. The delay introduced by video transmission in the network presents a difficulty. This delay creates a problem when we try to use information about eye movements for perceptual encoding. In this paper we discuss a new approach to the eye-tracker based video compression. Rather than relying on the point of gaze, this novel scheme tracks a vicinity of interest and offers a prediction mechanism for eye movements. The described system compensates the interim eye movements between the sampling and actual coding. The proposed scheme can be applied to a large variety of today's video compression standards. We have developed an eye gaze-aware MPEG-2 transcoder that can perceptually re-encode a live video stream in real time. The experiments we have conducted illustrate the substantial impact this integrated prediction method has on perceptual video compression and bit-rate reduction.