Currently, dynamic adaptive streaming over HTTP (DASH) standard is used to change the video resolution according to the network capacity of the end user’s. However, if the video resolution changes frequently, the attention of the users can be affected decreasing their quality-of-experience (QoE). In this context, this paper proposes a no-reference video quality metric that takes three impairment factors into account, initial buffering delay, temporal interruptions or pauses, and video resolution changes during a video transmission. Also, the temporal locations of pauses and resolution changes are considered. In order to perform this task, extensive subjective tests were conducted. Experimental results show that the evaluators’ QoE is highly correlated with the frequency of pauses, video resolution changes and initial delay. Based on these results a video streaming quality metric for DASH services (VsQM $_{\text {DASH}}$ ) is formulated and implemented on PC and mobile hand-held devices. Finally, the performance of VsQM $_{\text {DASH}}$ is validated with additional subjective tests reaching a person correlation coefficient of 0.99. Furthermore, experimental results demonstrated that the proposed metric has low complexity because it is based on application level parameters and its energy consumption is negligible in relation to the energy consumed by the video player; therefore, can be easily implemented in consumer electronic devices.

Video Quality Metric for Streaming Service Using DASH Standard

Video content providers put stringent requirements on the quality assessment methods realized on their services. They need to be accurate, real-time, adaptable to new content, and scalable as the video set grows. In this letter, we introduce a novel automated and computationally efficient video assessment method. It enables accurate real-time (online) analysis of delivered quality in an adaptable and scalable manner. Offline deep unsupervised learning processes are employed at the server side and inexpensive no-reference measurements at the client side. This provides both real-time assessment and performance comparable to the full reference counterpart, while maintaining its no-reference characteristics. We tested our approach on the LIMP Video Quality Database (an extensive packet loss impaired video set) obtaining a correlation between $78\%$ and $91\%$ to the FR benchmark (the video quality metric). Due to its unsupervised learning essence, our method is flexible and dynamically adaptable to new content and scalable with the number of videos.

/pdf/deep-learning-for-quality-assessment-in-live-video-streaming-2hpkyxmtsk.pdf

Deep Learning for Quality Assessment in Live Video Streaming

Recent years have seen increasing growth and popularity of gaming services, both interactive and passive. While interactive gaming video streaming applications have received much attention, passive gaming video streaming, in-spite of its huge success and growth in recent years, has seen much less interest from the research community. For the continued growth of such services in the future, it is imperative that the end user gaming quality of experience (QoE) is estimated so that it can be controlled and maximized to ensure user acceptance. Previous quality assessment studies have shown not so satisfactory performance of existing No-reference (NR) video quality assessment (VQA) metrics. Also, due to the inherent nature and different requirements of gaming video streaming applications, as well as the fact that gaming videos are perceived differently from non-gaming content (as they are usually computer generated and contain artificial/synthetic content), there is a need for application-specific light-weight, no-reference gaming video quality prediction models. In this paper, we present two NR machine learning-based quality estimation models for gaming video streaming, NR-GVSQI, and NR-GVSQE, using NR features, such as bitrate, resolution, and temporal information. We evaluate their performance on different gaming video datasets and show that the proposed models outperform the current state-of-the-art no-reference metrics, while also reaching a prediction accuracy comparable to the best known full reference metric.

/pdf/no-reference-video-quality-estimation-based-on-machine-3toh21ulgf.pdf

No-Reference Video Quality Estimation Based on Machine Learning for Passive Gaming Video Streaming Applications

With the exponential growth of video traffic over wireless networked and embedded devices, mechanisms are needed to predict and control perceptual video quality to meet the quality of experience (QoE) requirements in an energy-efficient way. This paper proposes an energy-efficient QoE support framework for wireless video communications. It consists of two components: 1) a perceptual video quality model that allows the prediction of video quality in real-time and with low complexity, and 2) an application layer energy-efficient and content-aware forward error correction (FEC) scheme for preventing quality degradation caused by network packet losses. The perceptual video quality model characterizes factors related to video content as well as distortion caused by compression and transmission. Prediction of perceptual quality is achieved through a decision tree using a set of observable features from the compressed bitstream and the network. The proposed model can achieve prediction accuracy of 88.9% and 90.5% on two distinct testing sets. Based on the proposed quality model, a novel FEC scheme is introduced to protect video packets from losses during transmission. Given a user-defined perceptual quality requirement, the FEC scheme adjusts the level of protection for different components in a video stream to minimize network overhead. Simulation results show that the proposed FEC scheme can enhance the perceptual quality of videos. Compared to conventional FEC methods for video communications, the proposed FEC scheme can reduce network overhead by 41% on average.

A Decision-Tree-Based Perceptual Video Quality Prediction Model and Its Application in FEC for Wireless Multimedia Communications

A method includes encoding, at a mobile device having a camera, a first portion of a media stream captured by the camera based on an encoding parameter. The method also includes sending the encoded first portion of the media stream from the mobile device to a computing device via a network connection. The method further includes detecting a change in available network bandwidth of the network connection. The method includes automatically adjusting the encoding parameter in response to the change in the available network bandwidth. The method also includes encoding a second portion of the media stream based on the adjusted encoding parameter and sending the encoded second portion of the media stream from the mobile device to the computing device.

Adjusting encoding parameters at a mobile device based on a change in available network bandwidth

In this work, we propose a novel no-reference (NR) video quality metric that evaluates the impact of frame freezing due to either packet loss or late arrival Our metric uses a trained neural network acting on features that are chosen to capture the impact of frame freezing on the perceived quality The considered features include the number of freezes, freeze duration statistics, inter-freeze distance statistics, frame difference before and after the freeze, normal frame difference, and the ratio of them We use the neural network to find the mapping between features and subjective test scores We optimize the network structure and the feature selection through a cross validation procedure, using training samples extracted from both VQEG and LIVE video databases The resulting feature set and network structure yields accurate quality prediction for both the training data containing 54 test videos and a separate testing dataset including 14 videos, with Pearson Correlation Coefficients greater than 09 and 08 for the training set and the testing set, respectively Our proposed metric has low complexity and could be utilized in a system with realtime processing constraint

A Novel No-reference Video Quality Metric for Evaluating Temporal Jerkiness due to Frame Freezing

In this work, we propose a novel no-reference (NR) video quality metric that evaluates the impact of frame freezing due to either packet loss or late arrival. Our metric uses a trained neural network acting on features that are chosen to capture the impact of frame freezing on the perceived quality. The considered features include the number of freezes, freeze duration statistics, inter-freeze distance statistics, frame difference before and after the freeze, normal frame difference, and the ratio of them. We use the neural network to find the mapping between features and subjective test scores. We optimize the network structure and the feature selection through a cross-validation procedure, using training samples extracted from both VQEG and LIVE video databases. The resulting feature set and network structure yields accurate quality prediction for both the training data containing 54 test videos and a separate testing dataset including 14 videos, with Pearson correlation coefficients greater than 0.9 and 0.8 for the training set and the testing set, respectively. Our proposed metric has low complexity and could be utilized in a system with real-time processing constraint.

http://vision.poly.edu/papers/arxiv/YuanyiXueArxivIP.pdf

A Novel No-Reference Video Quality Metric for Evaluating Temporal Jerkiness due to Frame Freezing

Improved systems and methods of performing multimedia communications over multimedia communications networks, in which video data senders can maintain high video quality of experience (QoE) levels with increased reliability despite changes in available bandwidths of video data receivers. In the disclosed systems and methods, video encoding parameters employed by the video data senders, including at least the video frame size and/or the video frame rate, can be dynamically adapted to the available bandwidths of the video data receivers, taking into account possible effects of spatial scaling and/or temporal scaling of video frames on the resulting video QoE.

Efficient approach to dynamic frame size and frame rate adaptation

Systems and methods of media servers for estimating the video quality of experience (QoE) of human viewers of video content or communications at video receivers. With these systems and methods, a media server can collect, within a plurality of predetermined time windows synchronized in time, operational parameters of ingress and egress networks over which such video content or communications can be transmitted, as well as coding parameters of a video decoder and a video encoder within the media server. Having collected the operational parameters of the ingress network and egress network and the coding parameters of the video decoder and video encoder, the media server can determine QoE scores associated with some or all the ingress network, the video decoder, the video encoder, and the egress network, and estimate a resulting video QoE for a human viewer at a video receiver as a predefined combination of the respective QoE scores.

Beril Erkin

Papers

A Novel No-reference Video Quality Metric for Evaluating Temporal Jerkiness due to Frame Freezing

A Novel No-Reference Video Quality Metric for Evaluating Temporal Jerkiness due to Frame Freezing

Efficient approach to dynamic frame size and frame rate adaptation

Estimation of video quality of experience on media services