Changing network conditions pose severe problems to video streaming in the Internet. HTTP adaptive streaming (HAS) is a technology employed by numerous video services that relieves these issues by adapting the video to the current network conditions. It enables service providers to improve resource utilization and Quality of Experience (QoE) by incorporating information from different layers in order to deliver and adapt a video in its best possible quality. Thereby, it allows taking into account end user device capabilities, available video quality levels, current network conditions, and current server load. For end users, the major benefits of HAS compared to classical HTTP video streaming are reduced interruptions of the video playback and higher bandwidth utilization, which both generally result in a higher QoE. Adaptation is possible by changing the frame rate, resolution, or quantization of the video, which can be done with various adaptation strategies and related client- and server-side actions. The technical development of HAS, existing open standardized solutions, but also proprietary solutions are reviewed in this paper as fundamental to derive the QoE influence factors that emerge as a result of adaptation. The main contribution is a comprehensive survey of QoE related works from human computer interaction and networking domains, which are structured according to the QoE impact of video adaptation. To be more precise, subjective studies that cover QoE aspects of adaptation dimensions and strategies are revisited. As a result, QoE influence factors of HAS and corresponding QoE models are identified, but also open issues and conflicting results are discussed. Furthermore, technical influence factors, which are often ignored in the context of HAS, affect perceptual QoE influence factors and are consequently analyzed. This survey gives the reader an overview of the current state of the art and recent developments. At the same time, it targets networking researchers who develop new solutions for HTTP video streaming or assess video streaming from a user centric point of view. Therefore, this paper is a major step toward truly improving HAS.

A Survey on Quality of Experience of HTTP Adaptive Streaming

High dynamic range (HDR) imaging techniques have been working constantly, actively, and validly in the fault detection and disease diagnosis in the astronomical and medical fields, and currently they have also gained much more attention from digital image processing and computer vision communities. While HDR imaging devices are starting to have friendly prices, HDR display devices are still out of reach of typical consumers. Due to the limited availability of HDR display devices, in most cases tone mapping operators (TMOs) are used to convert HDR images to standard low dynamic range (LDR) images for visualization. But existing TMOs cannot work effectively for all kinds of HDR images, with their performance largely depending on brightness, contrast, and structure properties of a scene. To accurately measure and compare the performance of distinct TMOs, in this paper develop an effective and efficient no-reference objective quality metric which can automatically assess LDR images created by different TMOs without access to the original HDR images. Our model is shown to be statistically superior to recent full- and no-reference quality measures on the existing tone-mapped image database and a new relevant database built in this work.

Blind Quality Assessment of Tone-Mapped Images Via Analysis of Information, Naturalness, and Structure

The field of perceptual quality assessment has gone through a wide range of developments and it is still growing. In particular, the area of no-reference (NR) image and video quality assessment has progressed rapidly during the last decade. In this article, we present a classification and review of latest published research work in the area of NR image and video quality assessment. The NR methods of visual quality assessment considered for review are structured into categories and subcategories based on the types of methodologies used for the underlying processing employed for quality estimation. Overall, the classification has been done into three categories, namely, pixel-based methods, bitstream-based methods, and hybrid methods of the aforementioned two categories. We believe that the review presented in this article will be helpful for practitioners as well as for researchers to keep abreast of the recent developments in the area of NR image and video quality assessment. This article can be used for various purposes such as gaining a structured overview of the field and to carry out performance comparisons for the state-of-the-art methods.

/pdf/no-reference-image-and-video-quality-assessment-a-46t3aha9ns.pdf

No-reference image and video quality assessment: a classification and review of recent approaches

Mulsemedia—multiple sensorial media—captures a wide variety of research efforts and applications This article presents a historic perspective on mulsemedia work and reviews current developments in the area These take place across the traditional multimedia spectrum—from virtual reality applications to computer games—as well as efforts in the arts, gastronomy, and therapy, to mention a few We also describe standardization efforts, via the MPEG-V standard, and identify future developments and exciting challenges the community needs to overcome

/pdf/mulsemedia-state-of-the-art-perspectives-and-challenges-4g3crhpl4g.pdf

Mulsemedia: State of the Art, Perspectives, and Challenges

This paper proposes a new exposure fusion approach for producing a high quality image result from multiple exposure images. Based on the local weight and global weight by considering the exposure quality measurement between different exposure images, and the just noticeable distortion-based saliency weight, a novel hybrid exposure weight measurement is developed. This new hybrid weight is guided not only by a single image's exposure level but also by the relative exposure level between different exposure images. The core of the approach is our novel boosting Laplacian pyramid, which is based on the structure of boosting the detail and base signal, respectively, and the boosting process is guided by the proposed exposure weight. Our approach can effectively blend the multiple exposure images for static scenes while preserving both color appearance and texture structure. Our experimental results demonstrate that the proposed approach successfully produces visually pleasing exposure fusion images with better color appearance and more texture details than the existing exposure fusion techniques and tone mapping operators.

Exposure fusion using boosting Laplacian pyramid.

Through years of development Content Networks (CN) have become more sophisticated and more technically diverse. Modern CN are designed to be more adaptive to communication environment, devices and user requirements. However, one open issue is the still fluctuating quality of service provision. As a result user experience can be negatively affected. In order to maintain a satisfactory level of user experience it is crucial to develop a feasible solution to measure the extent to which video services meet users' expectation. Assessing video quality with respect to users' subjective opinions is a complex task. In this paper we address challenges of this task and design an integrated framework using a number of comprehensive functional modules. Our framework integrates objective quality assessment models of Artifacts Measurement (AM) and Quality of Delivery (QoD) approaches. Only the fittest models are activated by the framework considering requirements of individual evaluation tasks. We also introduce our recent work of realising key functional modules of the framework. Joint subjective experiments between two institutes have also been carried out for the purpose of model implementation and evaluation. Results from experiments verify the concept of an integrated framework and show the effectiveness of its key modules in estimating the quality level of video services.

Framework for the integrated video quality assessment

The paper proposes a No-Reference (NR) metric to objectively assess the H.264/AVC video quality. The proposed model takes into account the typical artefacts introduced by hybrid block-based motion compensated predictive video codecs as the one related to the H.264/AVC standard. More specifically, these artefacts are the blockiness introduced at the boundaries of each coded block and the temporal flickering due to different coding modes used for the same macroblock along the video sequence. Furthermore, a flickering metric for intra coded frames is also derived. The quality prediction accuracy of the proposed NR quality metric is validated over subjective data collected during a video subjective evaluation experiments. Moreover, the quality prediction accuracy is also compared with the one provided by the well known state-of-the-art Structural SIMilarity (SSIM) metric which works in a full-reference mode. The proposed metric achieves a higher Pearson's correlation coefficient with subjective scores than the one achieved by the SSIM metric.

Perceptual quality assessment for H.264/AVC compression

Video transmission and analysis is often utilized in applications outside of the entertainment sector, and generally speaking this class of video is used to perform specific tasks. Examples of these applications include security and public safety. The Quality of Experience (QoE) concept for video content used for entertainment differs significantly from the QoE of surveillance video used for recognition tasks. This is because, in the latter case, the subjective satisfaction of the user depends on achieving a given functionality. Recognizing the growing importance of video in delivering a range of public safety services, we focused on developing critical quality thresholds in license plate recognition tasks based on videos streamed in constrained networking conditions. Since the number of surveillance cameras is still growing it is obvious that automatic systems will be used to do the tasks. Therefore, the presented research includes also analysis of automatic recognition algorithms.

/pdf/quality-assessment-for-a-visual-and-automatic-license-plate-215w1k7vqa.pdf

Quality assessment for a visual and automatic license plate recognition

Video bit rate reduction can be very important for all video streaming application. One of the possible ways to reduce bit rate is decreasing change in time or space domain i.e. changing frame rate or resolution. In this paper we present two no reference metrics mapping frame rate or resolution into MOS. Both models use simple to calculate parameters expressed by sequence spatial and temporal information. The models were estimated and verified upon distinctive video sequence sets. The considered frame rate change varies from 5 to 30 frames per second. The considered resolutions changes from SQCIF to SD.

QoE as a function of frame rate and resolution changes

Video transmission and analysis is often utilised in applications outside of the entertainment sector, and generally speaking this class of video is used to perform a specific task. Examples of these applications are security and public safety. The Quality of Experience (QoE) concept for video content used for entertainment differs significantly from the QoE of surveillance video used for recognition tasks. This is because, in the latter case, the subjective satisfaction of the user depends on achieving a given functionality. Moreover, such sequences have to be compressed significantly because the monitored place has to be seen on-line and it can be connected by an error prone wireless connection. Recognising the growing importance of video in delivering a range of public safety services, we focused on developing critical quality thresholds in licence plate recognition tasks based on videos streamed in constrained networking conditions.

Piotr Romaniak

Papers

Framework for the integrated video quality assessment

Perceptual quality assessment for H.264/AVC compression

Quality assessment for a visual and automatic license plate recognition

QoE as a function of frame rate and resolution changes

Quality Assessment for a Licence Plate Recognition Task Based on a Video Streamed in Limited Networking Conditions