Preference of Experience in Image Tone-Mapping: Dataset and Framework for Objective Measures Comparison
TL;DR: A subjective experiment attempting to determine users’ preference with respect to these two types of content in two different viewing scenarios—with and without the HDR reference shows that the absence of the reference can significantly influence the subjects' preferences for the natural images, while no significant impact has been found in the case of the synthetic images.
Abstract: The popularity of high dynamic range (HDR) imaging has grown in both academic and private research sectors. Since the native visualization of HDR content still has its limitations, the importance of dynamic range compression (i.e., tone-mapping) is very high. This paper evaluates observers’ preference of experience in context of image tone-mapping. Given the different nature of natural and computer-generated content, the way observers perceive the quality of tone-mapped images can be fundamentally different. In this paper, we describe a subjective experiment attempting to determine users’ preference with respect to these two types of content in two different viewing scenarios—with and without the HDR reference. The results show that the absence of the reference can significantly influence the subjects’ preferences for the natural images, while no significant impact has been found in the case of the synthetic images. Moreover, we introduce a benchmarking framework and compare the performance of selected objective metrics. The resulting dataset and framework are made publicly available to provide a common test bed and methodology for evaluating metrics in the considered scenario.
Citations
More filters
TL;DR: Experimental simulation results obtained from two large SCI databases have shown that the proposed GFM model yields a higher consistency with the human perception on the assessment of SCIs but also requires a lower computational complexity, compared with that of classical and state-of-the-art IQA models.
Abstract: In this paper, an accurate and efficient full-reference image quality assessment (IQA) model using the extracted Gabor features, called Gabor feature-based model (GFM), is proposed for conducting objective evaluation of screen content images (SCIs). It is well-known that the Gabor filters are highly consistent with the response of the human visual system (HVS), and the HVS is highly sensitive to the edge information. Based on these facts, the imaginary part of the Gabor filter that has odd symmetry and yields edge detection is exploited to the luminance of the reference and distorted SCI for extracting their Gabor features, respectively. The local similarities of the extracted Gabor features and two chrominance components, recorded in the LMN color space, are then measured independently. Finally, the Gabor-feature pooling strategy is employed to combine these measurements and generate the final evaluation score. Experimental simulation results obtained from two large SCI databases have shown that the proposed GFM model not only yields a higher consistency with the human perception on the assessment of SCIs but also requires a lower computational complexity, compared with that of classical and state-of-the-art IQA models. 1 1 The source code for the proposed GFM will be available at http://smartviplab.org/pubilcations/GFM.html .
93 citations
Additional excerpts
...[43]–[45] can not be employed to perform statistical significance test in this work....
[...]
TL;DR: This paper builds a compressed VR image quality (CVIQ) database, and proposes a multi-channel convolution neural network (CNN) for blind 360-degree image quality assessment (MC360IQA), which achieves the best performance among the state-of-art full-reference and no-reference image quality Assessment (IQA) models on the CVIQ database and other available360-degree IQA database.
Abstract: 360-degree images/videos have been dramatically increasing in recent years. The characteristic of omnidirectional-view results in high resolution of 360-degree images/videos, which makes them difficult to be transported and stored. To deal with the problem, video coding technologies are used to compress the omnidirectional content but they will introduce the compression distortion. Therefore, it is important to study how popular coding technologies affect the quality of 360-degree images. In this paper, we present a study on both subjective and objective quality assessment of compressed virtual reality (VR) images. We first build a compressed VR image quality (CVIQ) database including 16 reference images and 528 compressed ones with three prevailing coding technologies. Then, we propose a multi-channel convolution neural network (CNN) for blind 360-degree image quality assessment (MC360IQA). To be consistent with the visual content seen in the VR device, we project each 360-degree image into six viewport images, which are adopted as inputs of the proposed model. MC360IQA consists of two parts, a multi-channel CNN and an image quality regressor. The multi-channel CNN includes six parallel hyper-ResNet34 networks, where the hyper structure is used to incorporate the features from intermediate layers. The image quality regressor fuses the features and regresses them to final scores. The experimental results show that our model achieves the best performance among the state-of-art full-reference (FR) and no-reference (NR) image quality assessment (IQA) models on the CVIQ database and other available 360-degree IQA database.
91 citations
Cites background from "Preference of Experience in Image T..."
...[70]–[73], which evaluates classification abilities of IQA models to distinguish which of the two images is better or of the...
[...]
TL;DR: In this article, screen content, which is often computer-generated, has many characteristics distinctly different from conventional camera-captured natural scene content, and such characteristic differences impose majo...
Abstract: Screen content, which is often computer-generated, has many characteristics distinctly different from conventional camera-captured natural scene content. Such characteristic differences impose majo...
54 citations
TL;DR: The experimental results show that the proposed method can efficiently resist different TMOs and common image attacks, outperforming other existing HDR image watermarking methods.
25 citations
TL;DR: The focus of this contribution is the development of a QoE assessment framework, in line with the latest standardization progress in the field of QOE assessment, for understanding the visual effect of asymmetric and symmetric encoding for immersive media.
Abstract: The assessment of Quality of Experience (QoE) for stereoscopic 3-D video is a challenging task, especially in 3-D video compression and transmission applications. The focus of this contribution is the development of a QoE assessment framework, in line with the latest standardization progress in the field of QoE assessment, for understanding the visual effect of asymmetric and symmetric encoding for immersive media. Asymmetric stereoscopic video coding exploits the binocular suppression of the human vision system by representing one of the two views with a lower quality. This processing, while of limited effects on image quality, may influence the overall QoE. Many studies show that the QoE of immersive media such as 3-DTV can be thought as the combination of the perceived visual quality, the perceived depth quality, the visual fatigue, and visual discomfort. In this paper, we aim at: 1) exploiting the concept of preference of experience and protocols recently standardized for characterizing QoE; 2) conducting a case study using these standardized protocols to investigate the factors involving visual discomfort in stereoscopic video sequences with a focus on binocular rivalry; and 3) presenting the results of subjective experiments performed, by using the perceptual quality and preference of experience assessment protocols, for evaluating the impact of symmetrical, asymmetrical, and alternate coding schemes.
16 citations
Additional excerpts
...Preference of Experience (PoE) [15] and employed beyond 3DTV in context such as HDR QoE assessment [16]....
[...]
References
More filters
Proceedings Article•
13 Nov 2014
TL;DR: There is no statistical evidence to establish if the users preferred tone mapped content over the single exposure version as closer representation of the corresponding HDR scene, and the possible factors contributing to this somewhat unexpected finding are presented.
Abstract: Tone mapping operators (TMOs), employed to fit the dynamic range of High Dynamic Range (HDR) visual signals to that of the display, are generally non-transparent and modify the visual appearance of the scene. Despite this, tone mapped content generally tends to have more visual details as compared to a single exposure scene. It is however not clear if the extra details in tone mapped HDR affect user preferences over a single exposure content in terms of scene appearance and to what extent. This paper aims to shed light on this issue via a comprehensive subjective study. Our results reveal that there is no statistical evidence to establish if the users preferred tone mapped content over the single exposure version as closer representation of the corresponding HDR scene. We present those results as well as outline the possible factors contributing to this somewhat unexpected finding.
13 citations
26 May 2015
TL;DR: Whether the preference of the participating subjects is significantly affected by the presence of the original high dynamic range (HDR) image displayed on the HDR screen or if they can obtain equivalent results from the test without the reference is determined.
Abstract: The most reliable way to assess the quality of images after a dynamic range compression (i.e. tone-mapping) is a subjective study. The goal of this paper is to determine, whether the preference of the participating subjects is significantly affected by the presence of the original high dynamic range (HDR) image displayed on the HDR screen or if we can obtain equivalent results from the test without the reference. For that, an extensive experiment using two different setups has been performed. The statistical analysis showed significant difference between evaluations and a Monte Carlo simulation proved that the absence of the reference is the factor of influence.
12 citations
"Preference of Experience in Image T..." refers background in this paper
...These include both subjective evaluation and the development of objective methods....
[...]
01 Jan 2016
TL;DR: This chapter outlines a new dual modulation algorithm for the rendering of HDR content and discusses practical lessons that can be helpful toward therender of more accurate and visually appealing HDR content.
Abstract: High dynamic range (HDR) and wide color gamut video processing have started to receive significant research attention in recent years, especially in relation to content generation and processing. However, displaying such content is still a challenge, and efforts are required to make their deployment possible at consumer levels. While the emergence of locally backlit displays has helped to achieve higher contrast in the display of traditional low dynamic range videos, their use for displaying HDR videos is nontrivial. This chapter, therefore, aims to first identify a few challenges in this area. It then outlines a new dual modulation algorithm for the rendering of HDR content. It also discusses practical lessons that can be helpful toward the rendering of more accurate and visually appealing HDR content.
8 citations
TL;DR: This work investigates into a new objective method for TMO parameters optimization based on quantification of contrast reversal and naturalness that does not require any prior knowledge about the input HDR image and works independently on the used TMO.
Abstract: Dynamic range compression (or tone mapping) of HDR content is an essential step towards rendering it on traditional LDR displays in a meaningful way. This is however non-trivial and one of the reasons is that tone mapping operators (TMOs) usually need content-specific parameters to achieve the said goal. While subjective TMO parameter adjustment is the most accurate, it may not be easily deployable in many practical applications. Its subjective nature can also influence the comparison of different operators. Thus, there is a need for objective TMO parameter selection to automate the rendering process. To that end, we investigate into a new objective method for TMO parameters optimization. Our method is based on quantification of contrast reversal and naturalness. As an important advantage, it does not require any prior knowledge about the input HDR image and works independently on the used TMO. Experimental results using a variety of HDR images and several popular TMOs demonstrate the value of our method in comparison to default TMO parameter settings.
6 citations
"Preference of Experience in Image T..." refers result in this paper
...Hence, our studies provide valuable insights towards this goal....
[...]
TL;DR: This paper presents the universal method for TMO parameters tuning, in order to maintain as many details as possible, which is desirable in security applications, and suggests possible increase in privacy intrusion.
Abstract: High Dynamic Range (HDR) imaging has been gaining popularity in recent years. Different from the traditional low dynamic range (LDR), HDR content tends to be visually more appealing and realistic as it can represent the dynamic range of the visual stimuli present in the real world. As a result, more scene details can be faithfully reproduced. As a direct consequence, the visual quality tends to improve. HDR can be also directly exploited for new applications such as video surveillance and other security tasks. Since more scene details are available in HDR, it can help in identifying/tracking visual information which otherwise might be difficult with typical LDR content due to factors such as lack/excess of illumination, extreme contrast in the scene, etc. On the other hand, with HDR, there might be issues related to increased privacy intrusion. To display the HDR content on the regular screen, tone-mapping operators (TMO) are used. In this paper, we present the universal method for TMO parameters tuning, in order to maintain as many details as possible, which is desirable in security applications. The method’s performance is verified on several TMOs by comparing the outcomes from tone-mapping with default and optimized parameters. The results suggest that the proposed approach preserves more information which could be of advantage for security surveillance but, on the other hand, makes us consider possible increase in privacy intrusion.
6 citations
"Preference of Experience in Image T..." refers methods in this paper
...The combined method is called Tone-Mapped Images Quality Index (TMQI)....
[...]