Preference of Experience in Image Tone-Mapping: Dataset and Framework for Objective Measures Comparison
TL;DR: A subjective experiment attempting to determine users’ preference with respect to these two types of content in two different viewing scenarios—with and without the HDR reference shows that the absence of the reference can significantly influence the subjects' preferences for the natural images, while no significant impact has been found in the case of the synthetic images.
Abstract: The popularity of high dynamic range (HDR) imaging has grown in both academic and private research sectors. Since the native visualization of HDR content still has its limitations, the importance of dynamic range compression (i.e., tone-mapping) is very high. This paper evaluates observers’ preference of experience in context of image tone-mapping. Given the different nature of natural and computer-generated content, the way observers perceive the quality of tone-mapped images can be fundamentally different. In this paper, we describe a subjective experiment attempting to determine users’ preference with respect to these two types of content in two different viewing scenarios—with and without the HDR reference. The results show that the absence of the reference can significantly influence the subjects’ preferences for the natural images, while no significant impact has been found in the case of the synthetic images. Moreover, we introduce a benchmarking framework and compare the performance of selected objective metrics. The resulting dataset and framework are made publicly available to provide a common test bed and methodology for evaluating metrics in the considered scenario.
Citations
More filters
TL;DR: Experimental simulation results obtained from two large SCI databases have shown that the proposed GFM model yields a higher consistency with the human perception on the assessment of SCIs but also requires a lower computational complexity, compared with that of classical and state-of-the-art IQA models.
Abstract: In this paper, an accurate and efficient full-reference image quality assessment (IQA) model using the extracted Gabor features, called Gabor feature-based model (GFM), is proposed for conducting objective evaluation of screen content images (SCIs). It is well-known that the Gabor filters are highly consistent with the response of the human visual system (HVS), and the HVS is highly sensitive to the edge information. Based on these facts, the imaginary part of the Gabor filter that has odd symmetry and yields edge detection is exploited to the luminance of the reference and distorted SCI for extracting their Gabor features, respectively. The local similarities of the extracted Gabor features and two chrominance components, recorded in the LMN color space, are then measured independently. Finally, the Gabor-feature pooling strategy is employed to combine these measurements and generate the final evaluation score. Experimental simulation results obtained from two large SCI databases have shown that the proposed GFM model not only yields a higher consistency with the human perception on the assessment of SCIs but also requires a lower computational complexity, compared with that of classical and state-of-the-art IQA models. 1 1 The source code for the proposed GFM will be available at http://smartviplab.org/pubilcations/GFM.html .
93 citations
Additional excerpts
...[43]–[45] can not be employed to perform statistical significance test in this work....
[...]
TL;DR: This paper builds a compressed VR image quality (CVIQ) database, and proposes a multi-channel convolution neural network (CNN) for blind 360-degree image quality assessment (MC360IQA), which achieves the best performance among the state-of-art full-reference and no-reference image quality Assessment (IQA) models on the CVIQ database and other available360-degree IQA database.
Abstract: 360-degree images/videos have been dramatically increasing in recent years. The characteristic of omnidirectional-view results in high resolution of 360-degree images/videos, which makes them difficult to be transported and stored. To deal with the problem, video coding technologies are used to compress the omnidirectional content but they will introduce the compression distortion. Therefore, it is important to study how popular coding technologies affect the quality of 360-degree images. In this paper, we present a study on both subjective and objective quality assessment of compressed virtual reality (VR) images. We first build a compressed VR image quality (CVIQ) database including 16 reference images and 528 compressed ones with three prevailing coding technologies. Then, we propose a multi-channel convolution neural network (CNN) for blind 360-degree image quality assessment (MC360IQA). To be consistent with the visual content seen in the VR device, we project each 360-degree image into six viewport images, which are adopted as inputs of the proposed model. MC360IQA consists of two parts, a multi-channel CNN and an image quality regressor. The multi-channel CNN includes six parallel hyper-ResNet34 networks, where the hyper structure is used to incorporate the features from intermediate layers. The image quality regressor fuses the features and regresses them to final scores. The experimental results show that our model achieves the best performance among the state-of-art full-reference (FR) and no-reference (NR) image quality assessment (IQA) models on the CVIQ database and other available 360-degree IQA database.
91 citations
Cites background from "Preference of Experience in Image T..."
...[70]–[73], which evaluates classification abilities of IQA models to distinguish which of the two images is better or of the...
[...]
TL;DR: In this article, screen content, which is often computer-generated, has many characteristics distinctly different from conventional camera-captured natural scene content, and such characteristic differences impose majo...
Abstract: Screen content, which is often computer-generated, has many characteristics distinctly different from conventional camera-captured natural scene content. Such characteristic differences impose majo...
54 citations
TL;DR: The experimental results show that the proposed method can efficiently resist different TMOs and common image attacks, outperforming other existing HDR image watermarking methods.
25 citations
TL;DR: The focus of this contribution is the development of a QoE assessment framework, in line with the latest standardization progress in the field of QOE assessment, for understanding the visual effect of asymmetric and symmetric encoding for immersive media.
Abstract: The assessment of Quality of Experience (QoE) for stereoscopic 3-D video is a challenging task, especially in 3-D video compression and transmission applications. The focus of this contribution is the development of a QoE assessment framework, in line with the latest standardization progress in the field of QoE assessment, for understanding the visual effect of asymmetric and symmetric encoding for immersive media. Asymmetric stereoscopic video coding exploits the binocular suppression of the human vision system by representing one of the two views with a lower quality. This processing, while of limited effects on image quality, may influence the overall QoE. Many studies show that the QoE of immersive media such as 3-DTV can be thought as the combination of the perceived visual quality, the perceived depth quality, the visual fatigue, and visual discomfort. In this paper, we aim at: 1) exploiting the concept of preference of experience and protocols recently standardized for characterizing QoE; 2) conducting a case study using these standardized protocols to investigate the factors involving visual discomfort in stereoscopic video sequences with a focus on binocular rivalry; and 3) presenting the results of subjective experiments performed, by using the perceptual quality and preference of experience assessment protocols, for evaluating the impact of symmetrical, asymmetrical, and alternate coding schemes.
16 citations
Additional excerpts
...Preference of Experience (PoE) [15] and employed beyond 3DTV in context such as HDR QoE assessment [16]....
[...]
References
More filters
TL;DR: A new video database is presented: CVD2014-Camera Video Database, which uses real cameras rather than introducing distortions via post-processing, which results in a complex distortion space in regard to the video acquisition process.
Abstract: This paper presents a new database, CID2013, to address the issue of using no-reference (NR) image quality assessment algorithms on images with multiple distortions. Current NR algorithms struggle to handle images with many concurrent distortion types, such as real photographic images captured by different digital cameras. The database consists of six image sets; on average, 30 subjects have evaluated 12–14 devices depicting eight different scenes for a total of 79 different cameras, 480 images, and 188 subjects (67% female). The subjective evaluation method was a hybrid absolute category rating-pair comparison developed for the study and presented in this paper. This method utilizes a slideshow of all images within a scene to allow the test images to work as references to each other. In addition to mean opinion score value, the images are also rated using sharpness, graininess, lightness, and color saturation scales. The CID2013 database contains images used in the experiments with the full subjective data plus extensive background information from the subjects. The database is made freely available for the research community.
203 citations
TL;DR: It is shown that the appropriate choice of a tone-mapping operator (TMO) can significantly improve the reconstructed HDR quality and a statistical model is developed that approximates the distortion resulting from the combined processes of tone- mapping and compression.
Abstract: For backward compatible high dynamic range (HDR) video compression, the HDR sequence is reconstructed by inverse tone-mapping a compressed low dynamic range (LDR) version of the original HDR content. In this paper, we show that the appropriate choice of a tone-mapping operator (TMO) can significantly improve the reconstructed HDR quality. We develop a statistical model that approximates the distortion resulting from the combined processes of tone-mapping and compression. Using this model, we formulate a numerical optimization problem to find the tone-curve that minimizes the expected mean square error (MSE) in the reconstructed HDR sequence. We also develop a simplified model that reduces the computational complexity of the optimization problem to a closed-form solution. Performance evaluations show that the proposed methods provide superior performance in terms of HDR MSE and SSIM compared to existing tone-mapping schemes. It is also shown that the LDR image quality resulting from the proposed methods matches that produced by perceptually-based TMOs.
196 citations
"Preference of Experience in Image T..." refers result in this paper
...Hence, our studies provide valuable insights towards this goal....
[...]
TL;DR: The resulting algorithm, dubbed CurveletQA, correlates well with human subjective opinions of image quality, delivering performance that is competitive with popular full-reference IQA algorithms such as SSIM, and with top-performing NR IQA models.
Abstract: We study the efficacy of utilizing a powerful image descriptor, the curvelet transform, to learn a no-reference (NR) image quality assessment (IQA) model. A set of statistical features are extracted from a computed image curvelet representation, including the coordinates of the maxima of the log-histograms of the curvelet coefficients values, and the energy distributions of both orientation and scale in the curvelet domain. Our results indicate that these features are sensitive to the presence and severity of image distortion. Operating within a 2-stage framework of distortion classification followed by quality assessment, we train an image distortion and quality prediction engine using a support vector machine (SVM). The resulting algorithm, dubbed CurveletQA for short, was tested on the LIVE IQA database and compared to state-of-the-art NR/FR IQA algorithms. We found that CurveletQA correlates well with human subjective opinions of image quality, delivering performance that is competitive with popular full-reference (FR) IQA algorithms such as SSIM, and with top-performing NR IQA models. At the same time, CurveletQA has a relatively low complexity.
176 citations
Additional excerpts
...CurveletQA [20] or CS [21] have been introduced as well....
[...]
TL;DR: Numerical and subjective experiments demonstrate that the proposed algorithm consistently produces better quality tone mapped images even when the initial images of the iteration are created by the most competitive TMOs.
Abstract: Tone mapping operators (TMOs) aim to compress high dynamic range (HDR) images to low dynamic range (LDR) ones so as to visualize HDR images on standard displays. Most existing TMOs were demonstrated on specific examples without being thoroughly evaluated using well-designed and subject-validated image quality assessment models. A recently proposed tone mapped image quality index (TMQI) made one of the first attempts on objective quality assessment of tone mapped images. Here, we propose a substantially different approach to design TMO. Instead of using any predefined systematic computational structure for tone mapping (such as analytic image transformations and/or explicit contrast/edge enhancement), we directly navigate in the space of all images, searching for the image that optimizes an improved TMQI. In particular, we first improve the two building blocks in TMQI—structural fidelity and statistical naturalness components—leading to a TMQI-II metric. We then propose an iterative algorithm that alternatively improves the structural fidelity and statistical naturalness of the resulting image. Numerical and subjective experiments demonstrate that the proposed algorithm consistently produces better quality tone mapped images even when the initial images of the iteration are created by the most competitive TMOs. Meanwhile, these results also validate the superiority of TMQI-II over TMQI. 1 1 Partial preliminary results of this work were presented at ICASSP 2013 and ICME 2014.
133 citations
"Preference of Experience in Image T..." refers background in this paper
...Čadı́k et al. [5] also did not discover any statistical difference between experiments with and without real world reference....
[...]
29 Dec 2011
TL;DR: The resulting algorithm, which is named BLIINDS-II, requires minimal training and adopts a simple probabilistic model for score prediction, and is shown to correlate highly with human visual perception of quality, at a level that is even competitive with the powerful full-reference SSIM index.
Abstract: We propose an efficient, general-purpose, distortion-agnostic, blind/no-reference image quality assessment (NR-IQA) algorithm based on a natural scene statistics model of discrete cosine transform (DCT) coefficients. The algorithm is computationally appealing, given the availability of platforms optimized for DCT computation. We propose a generalized parametric model of the extracted DCT coefficients. The parameters of the model are utilized to predict image quality scores. The resulting algorithm, which we name BLIINDS-II, requires minimal training and adopts a simple probabilistic model for score prediction. When tested on the LIVE IQA database, BLIINDS-II is shown to correlate highly with human visual perception of quality, at a level that is even competitive with the powerful full-reference SSIM index.
121 citations
"Preference of Experience in Image T..." refers background in this paper
...One of the applications of the subjective experiments results is testing and training of objective metrics....
[...]