This paper reviews the evolution of video quality measurement techniques and their current state of the art. We start with subjective experiments and then discuss the various types of objective metrics and their uses. We also introduce V-Factor, a "hybrid" metric using both transport- and bitstream information. Finally, we summarize the main standardization activities, such as the work of the video quality experts group (VQEG), and we take a look at emerging trends in quality measurement, including image preference, visual attention, and audiovisual quality.

The Evolution of Video Quality Measurement: From PSNR to Hybrid Metrics

About the Author. Acknowledgements. Acronyms. 1 Introduction. 1.1 Motivation. 1.2 Outline. 2 Vision. 2.1 Eye. 2.2 Retina. 2.3 Visual Pathways. 2.4 Sensitivity to Light. 2.5 Color Perception. 2.6 Masking and Adaptation. 2.7 Multi-channel Organization. 2.8 Summary. 3 Video Quality. 3.1 Video Coding and Compression. 3.2 Artifacts. 3.3 Visual Quality. 3.4 Quality Metrics. 3.5 Metric Evaluation. 3.6 Summary. 4 Models and Metrics. 4.1 Isotropic Contrast. 4.2 Perceptual Distortion Metric. 4.3 Summary. 5 Metric Evaluation. 5.1 Still Images. 5.2 Video. 5.3 Component Analysis. 5.4 Summary. 6 Metric Extensions. 6.1 Blocking Artifacts. 6.2 Object Segmentation. 6.3 Image Appeal. 6.4 Summary. 7 Closing Remarks. 7.1 Summary. 7.2 Perspectives. Appendix: Color Space Conversions. References. Index.

Digital Video Quality: Vision Models and Metrics

An important and unsolved problem today is that of automatic quantification of the quality of video flows transmitted over packet networks. In particular, the ability to perform this task in real time (typically for streams sent themselves in real time) is especially interesting. The problem is still unsolved because there are many parameters affecting video quality, and their combined effect is not well identified and understood. Among these parameters, we have the source bit rate, the encoded frame type, the frame rate at the source, the packet loss rate in the network, etc. Only subjective evaluations give good results but, by definition, they are not automatic. We have previously explored the possibility of using artificial neural networks (NNs) to automatically quantify the quality of video flows and we showed that they can give results well correlated with human perception. In this paper, our goal is twofold. First, we report on a significant enhancement of our method by means of a new neural approach, the random NN model, and its learning algorithm, both of which offer better performances for our application. Second, we follow our approach to study and analyze the behavior of video quality for wide range variations of a set of selected parameters. This may help in developing control mechanisms in order to deliver the best possible video quality given the current network situation, and in better understanding of QoS aspects in multimedia engineering.

/pdf/a-study-of-real-time-packet-video-quality-using-random-51r5t26iyf.pdf

A study of real-time packet video quality using random neural networks

Objective metrics for measuring digital video performance are required by Government and industry for specification of system performance requirements, comparison of competing service offerings, service level agreements, network maintenance, and optimization of the use of limited network resources such as transmission bandwidth. To be accurate, digital video quality measurements must be based on the perceived quality of the actual video being received by the users of the digital video system rather than the measured quality of traditional video test signals (e.g., color bar). This is because the performance of digital video systems is variable and depends upon the dynamic characteristics of both the original video (e.g., spatial detail, motion) and the digital transmission system (e.g., bit rate, error rate). The goal of this report is to provide a complete description of the ITS video quality metric (VQM) algorithms and techniques. The ITS automated objective measurement algorithms provide close approximations to the overall quality impressions, or mean opinion scores, of digital video impairments that have been graded by panels of viewers.

Video Quality Measurement Techniques

Many organizations have focused on developing digital video quality metrics which produce results that accurately emulate subjective responses. However, to be widely applicable a metric must also work over a wide range of quality, and be useful for in-service quality monitoring. The Institute for Telecommunication Sciences (ITS) has developed spatial-temporal distortion metrics that meet all of these requirements. These objective metrics are described in detail and have a number of interesting properties, including utilization of 1) spatial activity filters which emphasize long edges on the order of 1/5 degree while simultaneously performing large amounts of noise suppression, 2) the angular direction of the spatial gradient, 3) spatial-temporal compression factors of at least 384:1 (spatial compression of at least 64:1 and temporal compression of at least 6:1, and 4) simple perceptibility thresholds and spatial-temporal masking functions. Results are presented that compare the objective metric values with mean opinion scores from a wide range of subjective data bases spanning many different scenes, systems, bit-rates, and applications.

Spatial-temporal distortion metric for in-service quality monitoring of any digital video system

This paper discusses the analysis of an audiovisual desktop video-teleconferencing subjective experiment conducted at the Institute for Telecommunication Sciences. Objective models of the individual audio and video quality are presented. Also discussed is an objective model of the audiovisual quality based upon the results of the individual objective audio and video quality models. Finally, a subjective model of audiovisual quality based upon users' ratings of the audio and video quality is discussed.

C. Jones

Papers

Development of opinion-based audiovisual quality models for desktop video-teleconferencing