The Information Theoretical Significance of Spatial and Temporal Masking in Video Signals

TLDR: In this article, a nonlinear spatio-temporal model of human threshold vision is proposed to predict both spatial and temporal masking effects accurately, and the maximum bit-rate savings due to masking are below 0.5 bit/sample in the average.

Abstract: We discuss the significance of masking effects for source coding of video signals without visible impairments. A new nonlinear spatiotemporal model of human threshold vision is proposed. Linearization yields the space-time-varying w-model. The model predicts both spatial and temporal masking effects accurately. Maximum bit-rate savings by irrelevancy reduction according to the w-model are evaluated for natural test pictures on the basis of the Shannon Lower Bound of rate distortion theory. Maximum bit-rate savings due to masking are below 0.5 bit/sample in the average. Typically 1/3 of the masking gain is due to spatial masking, the rest is due to the presence of dark and bright areas in the picture, where the visibility of noise is reduced. Gains due to temporal masking are significant only in the first 100 ms after a scene cut.