Video quality

About: Video quality is a research topic. Over the lifetime, 13143 publications have been published within this topic receiving 178307 citations.

Fast and robust search method for short video clips from large video collection

Abstract: In this paper a fast and robust method is proposed to search a large video collection for given short clips. Compared with existing video searching methods which use visual features only, our scheme performs a two-phase hierarchical matching technique using visual and audio features successively. Considering that video sampling rate (25 or 30 fps) is much lower than that of audio (8 to 48 kHz), a coarse search is implemented with sub-sampled video frames first, and then potential matches are verified and accurately located using fine audio features. Both features are extracted directly from MPEG compressed video for computational efficiency. Experiments have been conducted on over 10.5 hours of video to search for re-occurrences of 83 TV commercials and one news lead-out clip. All the 220 instances are correctly detected with no false alarm. Our experiments also show that the proposed method is robust to variations of video bit rate, frame rate, frame size and color shifting.

Providing Smoother Quality Layered Video Stream

Abstract: In recent years, one of the most popular Internet applications is web-based audio and video playback, where stored video is streamed from the server to a client on-demand. Rigid playback deadlines coupled with constraints on resources such as network bandwidth and client buffer make video delivery a challenging task [2]. These resources could be limited in such a way that it may not be possible to deliver full-quality video. In such a situation, it is desirable to minimize the degradation in the video quality while operating within the resource constraints [9]. Layered encoding is proposed to provide finer control on video quality: the video signal is split into layers and a subset of these layers is chosen such that the resource constraints are met [5]. However it is not a trivial task to select layers such that better but consistent quality playback is ensured when the network conditions are constantly varying. In our work, we address this layer selection problem in layered video delivery and show how smoother 1 quality video playback can be provided by utilizing the client buffer for prefetching. We first define smoothness criteria, design metrics for measuring it, and then develop off-line algorithms to maximize smoothness for the case where the network bandwidth is varying but known a priori. We then describe an adaptive algorithm for providing smoothed layered video delivery that doesn’t assume any knowledge about future bandwidth availability. The results of our experiments for measuring and comparing the performance these schemes are then presented. We conclude the paper with a brief discussion on our future work.

DeepQoE: A Multimodal Learning Framework for Video Quality of Experience (QoE) Prediction

Abstract: Recently, many models have been developed to predict video Quality of Experience (QoE), yet the applicability of these models still faces significant challenges. Firstly, many models rely on features that are unique to a specific dataset and thus lack the capability to generalize. Due to the intricate interactions among these features, a unified representation that is independent of datasets with different modalities is needed. Secondly, existing models often lack the configurability to perform both classification and regression tasks. Thirdly, the sample size of the available datasets to develop these models is often very small, and the impact of limited data on the performance of QoE models has not been adequately addressed. To address these issues, in this work we develop a novel and end-to-end framework termed as DeepQoE. The proposed framework first uses a combination of deep learning techniques, such as word embedding and 3D convolutional neural network (C3D), to extract generalized features. Next, these features are combined and fed into a neural network for representation learning. A learned representation will then serve as input for classification or regression tasks. We evaluate the performance of DeepQoE with three datasets. The results show that for small datasets (e.g., WHU-MVQoE2016 and Live-Netflix Video Database), the performance of state-of-the-art machine learning algorithms is greatly improved by using the QoE representation from DeepQoE (e.g., 35.71% to 44.82%); while for the large dataset (e.g., VideoSet), our DeepQoE framework achieves significant performance improvement in comparison to the best baseline method (90.94% vs. 82.84%). In addition to the much improved performance, DeepQoE has the flexibility to fit different datasets, to learn QoE representation, and to perform both classification and regression problems. We also develop a DeepQoE based adaptive bitrate streaming (ABR) system to verify that our framework can be easily applied to multimedia communication service. The software package of the DeepQoE framework has been released to facilitate the current research on QoE.

Receiver buffer requirement for video streaming over TCP

Abstract: TCP is one of the most widely used transport protocols for video streaming. However, the rate variability of TCP makes it difficult to provide good video quality. To accommodate the variability, video streaming applications require receiver-side buffering. In current practice, however, there are no systematic guidelines for the provisioning of the receiver buffer, and smooth playout is insured through over-provisioning. In this work, we are interested in memory-constrained applications where it is important to determine the right size of receiver buffer in order to insure a prescribed video quality. To that end, we characterize video streaming over TCP in a systematic and quantitative manner. We first model a video streaming system analytically and derive an expression of receiver buffer requirement based on the model. Our analysis shows that the receiver buffer requirement is determined by the network characteristics and desired video quality. Experimental results validate our model and demonstrate that the receiver buffer requirement achieves desired video quality.

Bandwidth-Efficient Multipath Transport Protocol for Quality-Guaranteed Real-Time Video Over Heterogeneous Wireless Networks

Abstract: Recent technological advancements in wireless infrastructures and handheld devices enable mobile users to concurrently receive multimedia contents with different radio interfaces (e.g., cellular and Wi-Fi). However, multipath video transport over the resource-limited and error-prone wireless networks is challenged with key technical issues: 1) conventional multipath protocols are throughput-oriented, and video data are scheduled in a content-agnostic fashion and 2) high-quality real-time video is bandwidth-intensive and delay-sensitive. To address these critical problems, this paper proposes a bandwidth-efficient multipath streaming (BEMA) protocol featured by the priority-aware data scheduling and forward error correction-based reliable transmission. First, we present a mathematical framework to formulate the delay-constrained distortion minimization problem for concurrent video transmission over multiple wireless access networks. Second, we develop a joint Raptor coding and data distribution framework to achieve target video quality with minimum bandwidth consumption. The proposed BEMA is able to effectively mitigate packet reordering and path asymmetry to improve network utilization. We conduct performance evaluation through extensive emulations in Exata involving real-time H.264 video streaming. Compared with the existing multipath protocols, BEMA achieves appreciable improvements in terms of video peak signal-to-noise ratio, end-to-end delay, bandwidth utilization, and goodput. Therefore, BEMA is recommended for streaming high-quality real-time video to multihomed terminals in heterogeneous wireless networks.