Benchmarking of objective quality metrics for HDR image quality assessment
02 Dec 2015-Eurasip Journal on Image and Video Processing (Springer International Publishing)-Vol. 2015, Iss: 1, pp 39
TL;DR: It is suggested that the performance of most full-reference metrics can be improved by considering non-linearities of the human visual system, while further efforts are necessary to improve performance of no-reference quality metrics for HDR content.
Abstract: Recent advances in high dynamic range (HDR) capture and display technologies have attracted a lot of interest from scientific, professional, and artistic communities. As in any technology, the evaluation of HDR systems in terms of quality of experience is essential. Subjective evaluations are time consuming and expensive, and thus objective quality assessment tools are needed as well. In this paper, we report and analyze the results of an extensive benchmarking of objective quality metrics for HDR image quality assessment. In total, 35 objective metrics were benchmarked on a database of 20 HDR contents encoded with 3 compression algorithms at 4 bit rates, leading to a total of 240 compressed HDR images, using subjective quality scores as ground truth. Performance indexes were computed to assess the accuracy, monotonicity, and consistency of the metric estimation of subjective scores. Statistical analysis was performed on the performance indexes to discriminate small differences between metrics. Results demonstrated that metrics designed for HDR content, i.e., HDR-VDP-2 and HDR-VQM, are the most reliable predictors of perceived quality. Finally, our findings suggested that the performance of most full-reference metrics can be improved by considering non-linearities of the human visual system, while further efforts are necessary to improve performance of no-reference quality metrics for HDR content.
Citations
More filters
01 Jan 2016
TL;DR: The digital video and hdtv algorithms and interfaces is universally compatible with any devices to read and is available in the book collection an online access to it is set as public so you can get it instantly.
Abstract: digital video and hdtv algorithms and interfaces is available in our book collection an online access to it is set as public so you can get it instantly. Our book servers saves in multiple countries, allowing you to get the most less latency time to download any of our books like this one. Merely said, the digital video and hdtv algorithms and interfaces is universally compatible with any devices to read.
219 citations
TL;DR: A comprehensive survey of the works that have been carried out over recent decades in perceptual audio, video, and joint audio-visual quality assessments is provided, describing existing methodologies in terms of requirement of a reference signal, feature extraction, feature mapping, and classification schemes.
Abstract: Measuring perceived quality of audio-visual signals at the end-user has become an important parameter in many multimedia networks and applications. It plays a crucial role in shaping audio-visual processing, compression, transmission and systems, along with their implementation, optimization, and testing. Service providers are enacting different quality of service (QoS) solutions to issue the best quality of experience (QoE) to their customers. Thus, devising precise perception-based quality metrics will greatly help improving multimedia services over wired and wireless networks. In this paper, we provide a comprehensive survey of the works that have been carried out over recent decades in perceptual audio, video, and joint audio-visual quality assessments, describing existing methodologies in terms of requirement of a reference signal, feature extraction, feature mapping, and classification schemes. In this context, an overview of quality formation and perception, QoS, QoE as well as quality of perception is also presented. Finally, open issues and challenges in audio-visual quality assessment are highlighted and potential future research directions are discussed.
70 citations
TL;DR: The first data hiding algorithm for OpenEXR HDR images offering a high embedding rate and producing high visual quality of the stego images is presented, and an adaptive data hiding approach for concealing more secret messages in pixels with low luminance is introduced.
Abstract: In this paper, we propose a novel data hiding algorithm for high dynamic range (HDR) images encoded by the OpenEXR file format. The proposed algorithm exploits each of three 10-bit mantissa fields as an embedding unit in order to conceal k bits of a secret message using an optimal base which produces the least pixel variation. An aggressive bit encoding and decomposition scheme is recommended, which offers a high probability to convey ( k + 1) bits without increasing the pixel variation caused by message concealment. In addition, we present a bit inversion embedding strategy to further increase the capacities when the probability of appearance of secret bit “1” is greater than 0.5. Furthermore, we introduce an adaptive data hiding approach for concealing more secret messages in pixels with low luminance, exploiting the features of the human visual system to achieve luminance-aware adaptive data hiding. The stego HDR images produced by our algorithm coincide with the HDR image file format, causing no suspicion from malicious eavesdroppers. The generated stego HDR images and their tone-mapped low dynamic range (LDR) images reveal no perceptual differences when subjected to quantitative testing by visual difference predictor. Our algorithm can resist steganalytic attacks from the HDR and LDR RS and SPAM steganalyzers. We present the first data hiding algorithm for OpenEXR HDR images offering a high embedding rate and producing high visual quality of the stego images. Our algorithm outperforms the current state-of-the-art works.
57 citations
Cites background from "Benchmarking of objective quality m..."
...We remark that while most of them were downloaded from the Internet available to the public [7], [31], [32], seven HDR images in the group-2 image database are derived from real scenes corresponding to the actual measured luminance....
[...]
...Later, Akyuz and Reinhard adopted the normalized log-average luminance of an HDR image to approximate the key value (ky ) of the scene [31] [32]....
[...]
TL;DR: The new ESPL-LIVE HDR Image Database is created containing diverse images obtained by tone-mapping operators and MEF algorithms, with and without post-processing, and a large-scale subjective study is conducted using a crowdsourced platform to gather more than 300 000 opinion scores.
Abstract: Measuring digital picture quality, as perceived by human observers, is increasingly important in many applications in which humans are the ultimate consumers of visual information. Standard dynamic range (SDR) images provide 8 b/color/pixel. High dynamic range (HDR) images, usually created from multiple exposures of the same scene, can provide 16 or 32 b/color/pixel, but need to be tonemapped to SDR for display on standard monitors. Multiexposure fusion (MEF) techniques bypass HDR creation by fusing an exposure stack directly to SDR images to achieve aesthetically pleasing luminance and color distributions. Many HDR and MEF databases have a relatively small number of images and human opinion scores, obtained under stringently controlled conditions, thereby limiting realistic viewing. Moreover, many of these databases are intended to compare tone-mapping algorithms, rather than being specialized for developing and comparing image quality assessment models. To overcome these challenges, we conducted a massively crowdsourced online subjective study. The primary contributions described in this paper are: 1) the new ESPL-LIVE HDR Image Database that we created containing diverse images obtained by tone-mapping operators and MEF algorithms, with and without post-processing; 2) a large-scale subjective study that we conducted using a crowdsourced platform to gather more than 300 000 opinion scores on 1811 images from over 5000 unique observers; and 3) a detailed study of the correlation performance of the state-of-the-art no-reference image quality assessment algorithms against human opinion scores of these images. The database is available at http://signal.ece.utexas.edu/%7Edebarati/HDRDatabase.zip .
55 citations
Cites methods from "Benchmarking of objective quality m..."
...[21] conducted a subjective experiment using 240 images obtained by tonemapping 20 HDR images with a display adaptive tone-mapping algorithm and compressing them using different profiles of the JPEG XT [23] compression algorithm....
[...]
...“Next Image” button, the position of the slider was converted to an integer valued quality score between [1-100] and the...
[...]
03 Apr 2017
TL;DR: This paper gathers several existing HDR image databases with subjective quality annotations, and analyzes in depth many FR metrics, including those used in MPEG standardization, using both classical correlation analyses and classification accuracy.
Abstract: High dynamic range (HDR) image and video technology has recently attracted a great deal of attention in the multimedia community, as a mean to produce truly realistic video and further improve the quality of experience (QoE) of emerging multimedia services. In this context, measuring the quality of compressed HDR content plays a fundamental role. However, full-reference (FR) HDR visual quality assessment poses new challenges with respect to the conventional low dynamic range case. Quality metrics have to be redesigned or adapted to HDR, and understanding their reliability to predict users’ judgments is even more critical due to the still limited availability of HDR displays to perform subjective evaluations. The goal of this paper is to provide a complete and thorough survey of the performance of the most popular HDR FR image quality metrics. To this end, we gather several existing HDR image databases with subjective quality annotations, in addition to a new one created by ourselves. After aligning the scores in these databases, we obtain an extensive set of 690 compressed HDR images, along with their subjective quality. Next, we analyze in depth many FR metrics, including those used in MPEG standardization, using both classical correlation analyses and classification accuracy. We believe that our results could serve as the most complete and comprehensive benchmark of image quality metrics in the field of HDR image compression.
42 citations
Cites background or result from "Benchmarking of objective quality m..."
...In other cases, such as [17], metrics have been tested on a single type of distortion only (specifically JPEG-XT compression), thus it is desirable to extend those conclusions to more realistic and variegate conditions....
[...]
...We also confirm the findings in previous work [17, 59] that legacy LDR image quality metrics have good prediction and discrimination performance, provided that a proper transformation such as PU encoding is done beforehand....
[...]
...The capability of both kinds of fidelity metrics to predict viewers’ mean opinion scores (MOS) has been assessed in a number of recent subjective studies using compressed HDR pictures [17, 37, 39, 59]....
[...]
References
More filters
TL;DR: In this article, a structural similarity index is proposed for image quality assessment based on the degradation of structural information, which can be applied to both subjective ratings and objective methods on a database of images compressed with JPEG and JPEG2000.
Abstract: Objective methods for assessing perceptual image quality traditionally attempted to quantify the visibility of errors (differences) between a distorted image and a reference image using a variety of known properties of the human visual system. Under the assumption that human visual perception is highly adapted for extracting structural information from a scene, we introduce an alternative complementary framework for quality assessment based on the degradation of structural information. As a specific example of this concept, we develop a structural similarity index and demonstrate its promise through a set of intuitive examples, as well as comparison to both subjective ratings and state-of-the-art objective methods on a database of images compressed with JPEG and JPEG2000. A MATLAB implementation of the proposed algorithm is available online at http://www.cns.nyu.edu//spl sim/lcv/ssim/.
40,609 citations
TL;DR: Although the new index is mathematically defined and no human visual system model is explicitly employed, experiments on various image distortion types indicate that it performs significantly better than the widely used distortion metric mean squared error.
Abstract: We propose a new universal objective image quality index, which is easy to calculate and applicable to various image processing applications. Instead of using traditional error summation methods, the proposed index is designed by modeling any image distortion as a combination of three factors: loss of correlation, luminance distortion, and contrast distortion. Although the new index is mathematically defined and no human visual system model is explicitly employed, our experiments on various image distortion types indicate that it performs significantly better than the widely used distortion metric mean squared error. Demonstrative images and an efficient MATLAB implementation of the algorithm are available online at http://anchovy.ece.utexas.edu//spl sim/zwang/research/quality_index/demo.html.
5,285 citations
09 Nov 2003
TL;DR: This paper proposes a multiscale structural similarity method, which supplies more flexibility than previous single-scale methods in incorporating the variations of viewing conditions, and develops an image synthesis method to calibrate the parameters that define the relative importance of different scales.
Abstract: The structural similarity image quality paradigm is based on the assumption that the human visual system is highly adapted for extracting structural information from the scene, and therefore a measure of structural similarity can provide a good approximation to perceived image quality. This paper proposes a multiscale structural similarity method, which supplies more flexibility than previous single-scale methods in incorporating the variations of viewing conditions. We develop an image synthesis method to calibrate the parameters that define the relative importance of different scales. Experimental comparisons demonstrate the effectiveness of the proposed method.
4,333 citations
TL;DR: A novel feature similarity (FSIM) index for full reference IQA is proposed based on the fact that human visual system (HVS) understands an image mainly according to its low-level features.
Abstract: Image quality assessment (IQA) aims to use computational models to measure the image quality consistently with subjective evaluations. The well-known structural similarity index brings IQA from pixel- to structure-based stage. In this paper, a novel feature similarity (FSIM) index for full reference IQA is proposed based on the fact that human visual system (HVS) understands an image mainly according to its low-level features. Specifically, the phase congruency (PC), which is a dimensionless measure of the significance of a local structure, is used as the primary feature in FSIM. Considering that PC is contrast invariant while the contrast information does affect HVS' perception of image quality, the image gradient magnitude (GM) is employed as the secondary feature in FSIM. PC and GM play complementary roles in characterizing the image local quality. After obtaining the local quality map, we use PC again as a weighting function to derive a single quality score. Extensive experiments performed on six benchmark IQA databases demonstrate that FSIM can achieve much higher consistency with the subjective evaluations than state-of-the-art IQA metrics.
4,028 citations
TL;DR: An image information measure is proposed that quantifies the information that is present in the reference image and how much of this reference information can be extracted from the distorted image and combined these two quantities form a visual information fidelity measure for image QA.
Abstract: Measurement of visual quality is of fundamental importance to numerous image and video processing applications. The goal of quality assessment (QA) research is to design algorithms that can automatically assess the quality of images or videos in a perceptually consistent manner. Image QA algorithms generally interpret image quality as fidelity or similarity with a "reference" or "perfect" image in some perceptual space. Such "full-reference" QA methods attempt to achieve consistency in quality prediction by modeling salient physiological and psychovisual features of the human visual system (HVS), or by signal fidelity measures. In this paper, we approach the image QA problem as an information fidelity problem. Specifically, we propose to quantify the loss of image information to the distortion process and explore the relationship between image information and visual quality. QA systems are invariably involved with judging the visual quality of "natural" images and videos that are meant for "human consumption." Researchers have developed sophisticated models to capture the statistics of such natural signals. Using these models, we previously presented an information fidelity criterion for image QA that related image quality with the amount of information shared between a reference and a distorted image. In this paper, we propose an image information measure that quantifies the information that is present in the reference image and how much of this reference information can be extracted from the distorted image. Combining these two quantities, we propose a visual information fidelity measure for image QA. We validate the performance of our algorithm with an extensive subjective study involving 779 images and show that our method outperforms recent state-of-the-art image QA algorithms by a sizeable margin in our simulations. The code and the data from the subjective study are available at the LIVE website.
3,146 citations