Video quality assessment using temporal quality variations and machine learning
11 Jul 2011-pp 1-6
TL;DR: Experiments conducted using two publicly available video databases show the effectiveness of the proposed full-reference metric in comparison to the relevant existing VQA metrics.
Abstract: Objective video quality assessment (VQA) is the use of computational models to predict the video quality in line with the perception of the human visual system (HVS). It is challenging due to the underlying complexity, and the relatively limited understanding of the HVS and its intricate mechanisms. There are two important issues regarding VQA: (a) the temporal factors apart from the spatial ones also need to be considered, (b) the contribution of each factor and their interaction to the overall video quality needs to be determined. In this paper, we attempt to tackle the first issue by utilizing the variation of spatial quality along the temporal axis. The second issue is addressed by the use of machine learning; we believe this to be more convincing since the relationship between the factors and the overall quality is derived via training with substantial ground truth (i.e. subjective scores). Experiments conducted using two publicly available video databases show the effectiveness of the proposed full-reference metric in comparison to the relevant existing VQA metrics.
Citations
More filters
01 Jan 2013
TL;DR: This work provides an in-depth review of recent developments in the field of visual quality assessment and puts equal emphasis on video quality databases and metrics as this is a less investigated area.
Abstract: Research on visual quality assessment has been active during the last decade. In this work, we provide an in-depth review of recent developments in the field. As compared with existing survey papers, our current work has several unique contributions. First, besides image quality databases and metrics, we put equal emphasis on video quality databases and metrics as this is a less investigated area. Second, we discuss the application of visual quality evaluation to perceptual coding as an example for applications. Third, we benchmark the performance of state-of-the-art visual quality metrics with experiments. Finally, future trends in visual quality assessment are discussed.
63 citations
TL;DR: This work focuses on recent progresses of quality metrics, which have been reviewed for the newly emerged forms of visual signals, which include scalable and mobile videos, High Dynamic Range (HDR) images, image segmentation results, 3D images/videos, and retargeted images.
Abstract: While quality assessment is essential for testing, optimizing, benchmarking, monitoring, and inspecting related systems and services, it also plays an essential role in the design of virtually all visual signal processing and communication algorithms, as well as various related decision-making processes. In this paper, we first provide an overview of recently derived quality assessment approaches for traditional visual signals (i.e., 2D images/videos), with highlights for new trends (such as machine learning approaches). On the other hand, with the ongoing development of devices and multimedia services, newly emerged visual signals (e.g., mobile/3D videos) are becoming more and more popular. This work focuses on recent progresses of quality metrics, which have been reviewed for the newly emerged forms of visual signals, which include scalable and mobile videos, High Dynamic Range (HDR) images, image segmentation results, 3D images/videos, and retargeted images.
16 citations
Additional excerpts
...After the review, two main trends of quality assessment are proposed for FR visual signal quality assessment, which are impairment decoupling [11, 22-27] and machine learning approaches [28-34], respectively....
[...]
23 Jul 2018
TL;DR: The extensive experiments in the LIVE Video Quality Database suggest the proposed video quality assessment model has superior correlation performance with human visual perception than other state-of-the-art methods.
Abstract: In this work, we proposed a full-reference method to estimate video quality. First, we decompose the video into one spatial image and two spatiotemporal slice images. Then for each one of them, sixteen Laws texture filters are applied to generate nine different Laws feature maps. In order to compare the similarity degree of these feature maps obtained from both original and distorted videos, we compute the twodimensional correlation coefficients. Since the correlation coefficients are computed for each frame and spatiotemporal slice, we only choose four statistical values to represent them to reduce the complexity. Lastly, the regression approach is chosen to learn the mapping relationship between the selected features and subjective quality scores. The extensive experiments in the LIVE Video Quality Database suggest our proposed video quality assessment model has superior correlation performance with human visual perception than other state-of-the-art methods.
9 citations
Cites methods from "Video quality assessment using temp..."
...In [14], the authors used the variation of quality along time axis as the temporal factor....
[...]
Proceedings Article•
01 Dec 2012TL;DR: This work decomposes an input video clip into multiple smaller intervals, measure the quality of each interval separately, and applies a fusion approach to integrating these scores into a final one to improve MOVIE and is also competitive with other state-of-the-art video quality metrics.
Abstract: In this work, we decompose an input video clip into multiple smaller intervals, measure the quality of each interval separately, and apply a fusion approach to integrating these scores into a final one. To give more details, an input video clip is first decomposed into smaller units along the temporal domain, called the temporal decomposition units (TDUs). Next, for each TDU that consists of a small number of frames, we adopt a proper video quality metric (specifically, the MOVIE index in this work) to compute the quality scores of all frames and, based on the sociological findings, choose the worst scores of TDUs for data fusion. Finally, a regression approach is used to fuse selected worst scores from all TDUs to get the ultimate quality score of the input video as a whole. We conduct extensive experiments on the LIVE video database, and show that the proposed approach indeed improves MOVIE and is also competitive with other state-of-the-art video quality metrics.
7 citations
Dissertation•
24 May 2012
TL;DR: A series of novel image classification algorithms based on CW-SSIM, which does not involve any registration, intensity normalization or sophisticated feature extraction processes, and does not rely on any modeling of the image patterns or distortion processes, achieves competitive performance with reduced computational cost.
Abstract: Complex wavelet structural similarity (CW-SSIM) index has been recognized as a novel image similarity measure of broad potential applications due to its robustness to small geometric distortions such as translation, scaling and rotation of images. Nevertheless, how to make the best use of it in image classification problems has not been deeply investigated. In this study, we introduce a series of novel image classification algorithms based on CW-SSIM and use handwritten digit and face image recognition as examples for demonstration, including CW-SSIM based nearest neighbor method, CW-SSIM based k means method, CW-SSIM based support vector machine method (SVM) and CW-SSIM based SVM using affinity propagation. Among the proposed approaches, the best compromise between accuracy and complexity is obtained by the CW-SSIM support vector machine algorithm, which combines an unsupervised clustering method to divide the training images into clusters with representative images and a supervised learning method based on support vector machines to maximize the classification accuracy. Our experiments show that such a conceptually simple image classification method, which does not involve any registration, intensity normalization or sophisticated feature extraction processes, and does not rely on any modeling of the image patterns or distortion processes, achieves competitive performance with reduced computational cost.
3 citations
Cites background from "Video quality assessment using temp..."
...In [39], Narwaria and Lin stated that the matrix UV T can be interpreted as the ensemble of the basis images, whereas the singular values σ are the weights assigned to these basis images....
[...]
References
More filters
14 May 2006
TL;DR: This work describes some innovative guidelines for easy and reliable determination of a quality metric that is subjectively meaningful: mean time between failures (MTBF), representing how often a typical viewer observes a noticeable visual error and a related instantaneous metric relating to the fraction of viewers that find a given video portion to be within acceptable quality levels.
Abstract: As digital communication of television content becomes more pervasive, the long-standing problem of assessing video quality becomes particularly important. This work describes some innovative guidelines for easy and reliable determination of a quality metric that is subjectively meaningful: mean time between failures (MTBF), representing how often a typical viewer observes a noticeable visual error and a related instantaneous metric relating to the fraction of viewers that find a given video portion to be within acceptable quality levels. The value of MTBF is addressed in the context of video quality, and objective measurements that correlate well with subjective evaluations of MTBF are investigated for different video clips at bit rates in the range of 1.5 - 5 Mbps. The full-reference objective metrics of PSNR and JND are found to have correlation coefficients of 0.69 and 0.88 respectively. In contrast, a reduced-reference objective metric, spatial temporal join metric (S. Wolf and M.H. Pinson, 1999) (STJM) is found to have a correlation coefficient of 0.84. A method for estimating the MTBF of a video sequence from objective measurements is also described
17 citations
"Video quality assessment using temp..." refers background or methods or result in this paper
...Based upon the analysis and reasoning to be presented in this work, we use the variation of spatial quality in time as a measure of quality fluctuations since it has impact [16], [18]-[20] on the perceived video quality....
[...]
...We compared the proposed metric QSVR with some of the existing metrics namely the widely used PSNR, SpeedSSIM [11], V-VIF [16] and the Video Quality Metric (VQM) [12]....
[...]
...Firstly, unlike images, the perceived video quality depends not only on the spatial or frame-level quality but can also be affected by the quality variation along the temporal axis [6]-[12], [15]-[16], [18], [20]....
[...]
...As has been noted by many researchers, considering quality along the temporal axis is an important factor for VQA [6]-[12], [15]-[16], [18], [20]....
[...]
...The authors in [16] proposed a method known as Mean Time Between Failures (MTBF) in which the viewer continuously indicates the presence of perceptual artifacts (like blockiness, blurriness) in the video sequence by using a buzzer....
[...]
21 Jun 2010
TL;DR: Results prove the existence of a purely temporal aspect in video quality perception and compare the perceived quality of two versions of a mosquito noise correction algorithm: one purely spatial and the other spatio-temporal.
Abstract: For quality assessment, videos are often considered as series of images with, at best, a motion component. To study the role of temporal aspects in quality, we compare the perceived quality of two versions of a mosquito noise correction algorithm: one purely spatial and the other spatio-temporal. We set up a paired-comparison experiment specially adapted to the temporal aspects of video quality. Results prove the existence of a purely temporal aspect in video quality perception.
8 citations
"Video quality assessment using temp..." refers background or methods in this paper
...Therefore, variation of quality in time is an important factor in VQA [16], [18]-[20]....
[...]
...As has been noted by many researchers, considering quality along the temporal axis is an important factor for VQA [6]-[12], [15]-[16], [18], [20]....
[...]
...Based upon the analysis and reasoning to be presented in this work, we use the variation of spatial quality in time as a measure of quality fluctuations since it has impact [16], [18]-[20] on the perceived video quality....
[...]
...Firstly, unlike images, the perceived video quality depends not only on the spatial or frame-level quality but can also be affected by the quality variation along the temporal axis [6]-[12], [15]-[16], [18], [20]....
[...]