On the Minimum Perceptual Temporal Video Sampling Rate and Its Application to Adaptive Frame Skipping
01 Sep 2019-IEEE Transactions on Circuits and Systems for Video Technology (Institute of Electrical and Electronics Engineers (IEEE))-Vol. 29, Iss: 9, pp 2760-2774
TL;DR: This work determines the minimum temporal sampling rate at which a video should be presented to make temporal sampling imperceptible to viewers, and proposes a model to compute the required minimum sampling rate based on these two parameters.
Abstract: Media technology, in particular video recording and playback, keeps improving to provide users with high-quality real and virtual visual content. In recent years, increasing the temporal sampling rate of videos and the refresh rate of displays has become one focus of technical innovation. This raises the question, how high the sampling and refresh rates should be? To answer this question, we determine the minimum temporal sampling rate at which a video should be presented to make temporal sampling imperceptible to viewers. Through a psychophysical study, we find that this minimum sampling rate depends on both the speed of the objects in the image plane and the exposure time of the recording camera. We propose a model to compute the required minimum sampling rate based on these two parameters. In addition, state-of-the-art video codecs employ motion vectors from which the local object movement speed can be inferred. Therefore, we present a procedure to compute the minimum sampling rate given an encoded video and camera exposure time. Since the object motion speed in a video may vary, the corresponding minimum frame rate is also varying. This is why the results of this paper are particularly applicable when used together with adaptive frame rate computer generated graphics or novel video communication solutions that drop insignificant frames. In our experiments, we show that videos played back at the minimum adaptive frame rate achieve an average bit rate reduction of 26% compared to constant frame rate playback, while perceptually no difference can be observed.
References
More filters
••
TL;DR: An overview of the technical features of H.264/AVC is provided, profiles and applications for the standard are described, and the history of the standardization process is outlined.
Abstract: H.264/AVC is newest video coding standard of the ITU-T Video Coding Experts Group and the ISO/IEC Moving Picture Experts Group. The main goals of the H.264/AVC standardization effort have been enhanced compression performance and provision of a "network-friendly" video representation addressing "conversational" (video telephony) and "nonconversational" (storage, broadcast, or streaming) applications. H.264/AVC has achieved a significant improvement in rate-distortion efficiency relative to existing standards. This article provides an overview of the technical features of H.264/AVC, describes profiles and applications for the standard, and outlines the history of the standardization process.
8,646 citations
"On the Minimum Perceptual Temporal ..." refers background or methods in this paper
...264 [13] video codec), we can obtain the absolute frame rate threshold fc for each video frame....
[...]
...264 [13] and writes it to a file for later presentation to the users....
[...]
•
01 Jan 1965
TL;DR: In this paper, the authors provide a broad overview of Fourier Transform and its relation with the FFT and the Hartley Transform, as well as the Laplace Transform and the Laplacian Transform.
Abstract: 1 Introduction 2 Groundwork 3 Convolution 4 Notation for Some Useful Functions 5 The Impulse Symbol 6 The Basic Theorems 7 Obtaining Transforms 8 The Two Domains 9 Waveforms, Spectra, Filters and Linearity 10 Sampling and Series 11 The Discrete Fourier Transform and the FFT 12 The Discrete Hartley Transform 13 Relatives of the Fourier Transform 14 The Laplace Transform 15 Antennas and Optics 16 Applications in Statistics 17 Random Waveforms and Noise 18 Heat Conduction and Diffusion 19 Dynamic Power Spectra 20 Tables of sinc x, sinc2x, and exp(-71x2) 21 Solutions to Selected Problems 22 Pictorial Dictionary of Fourier Transforms 23 The Life of Joseph Fourier
5,714 citations
••
TL;DR: A broad class of up‐down methods used in psychoacoustics with due emphasis on the related problems of parameter estimation and the efficient placing of observations is described, including examples where conventional techniques are inapplicable.
Abstract: During the past decade a number of variations in the simple up‐down procedure have been used in psychoacoustic testing. A broad class of these methods is described with due emphasis on the related problems of parameter estimation and the efficient placing of observations. The advantages of up‐down methods are many, including simplicity, high efficiency, robustness, small‐sample reliability, and relative freedom from restrictive assumptions. Several applications of these procedures in psychoacoustics are described, including examples where conventional techniques are inapplicable.
5,306 citations
"On the Minimum Perceptual Temporal ..." refers methods in this paper
...To avoid this issue, we used a modified variant of the simple up-down or staircase method [15], [16]....
[...]
••
TL;DR: In this article, the first stage consists of linear filters that are oriented in space-time and tuned in spatial frequency, and the outputs of quadrature pairs of such filters are squared and summed to give a measure of motion energy.
Abstract: A motion sequence may be represented as a single pattern in x–y–t space; a velocity of motion corresponds to a three-dimensional orientation in this space. Motion sinformation can be extracted by a system that responds to the oriented spatiotemporal energy. We discuss a class of models for human motion mechanisms in which the first stage consists of linear filters that are oriented in space-time and tuned in spatial frequency. The outputs of quadrature pairs of such filters are squared and summed to give a measure of motion energy. These responses are then fed into an opponent stage. Energy models can be built from elements that are consistent with known physiology and psychophysics, and they permit a qualitative understanding of a variety of motion phenomena.
3,504 citations
••
2,834 citations