Evaluating Quality of Experience for Streaming Video in Real Time

TLDR: A scalable, lightweight, no-reference framework to infer video QoE, which shows that the MOS predictions are in close agreement with subjective perceptions and an implementation of the framework on standard Linux PC shows it can compute 20 MOS calculations per second with 3 parameters and 18 partitions of the QeE space.

Abstract: We present a scalable, lightweight, no-reference framework to infer video QoE. Our framework revolves around a one time offline construction of a k-dimensional space, which we call the QoE space. The k-dimensions accommodate k parameters (network-dependent/independent) that potentially affect video quality. The k-dimensional space is partitioned to N representative zones, each with a QoE index. Instantaneous parameter values are matched with the indices to infer QoE. To validate our framework, we construct a 3-dimensional QoE space with bit-rate, loss, and delay as the principal components. We create 18 video samples with unique combinations of the 3 parameters. 77 human subjects rated these video samples on a scale of 1 to 5 to create the QoE space. In a second set of survey, our predicted MOS was compared to 49 human responses. Results show that our MOS predictions are in close agreement with subjective perceptions. An implementation of our framework on standard Linux PC shows we can compute 20 MOS calculations per second with 3 parameters and 18 partitions of the QoE space.