With the emergence of high-dynamic range (HDR) imaging, the existing visual signal processing systems will need to deal with both HDR and standard dynamic range (SDR) signals. In such systems, computing the objective quality is an important aspect in various optimization processes (e.g., video encoding). To that end, we present a newly calibrated objective method that can tackle both HDR and SDR signals. As it is based on the previously proposed HDR-VDP-2 method, we refer to the newly calibrated metric as HDR-VDP-2.2. Our main contribution is toward improving the frequency-based pooling in HDR-VDP-2 to enhance its objective quality prediction accuracy. We achieve this by formulating and solving a constrained optimization problem and thereby finding the optimal pooling weights. We also carried out extensive cross-validation as well as verified the performance of the new method on independent databases. These indicate clear improvement in prediction accuracy as compared with the default pooling weights. The source codes for HDR-VDP-2.2 are publicly available online for free download and use.

/pdf/hdr-vdp-2-2-a-calibrated-method-for-objective-quality-3tvwsqkpk0.pdf

HDR-VDP-2.2: a calibrated method for objective quality prediction of high-dynamic range and standard images

High dynamic range (HDR) signals fundamentally differ from the traditional low dynamic range (LDR) ones in that pixels are related (proportional) to the physical luminance in the scene (i.e. scene-referred). For that reason, the existing LDR video quality measurement methods may not be directly used for assessing quality in HDR videos. To address that, we present an objective HDR video quality measure (HDR-VQM) based on signal pre-processing, transformation, and subsequent frequency based decomposition. Video quality is then computed based on a spatio-temporal analysis that relates to human eye fixation behavior during video viewing. Consequently, the proposed method does not involve expensive computations related to explicit motion analysis in the HDR video signal, and is therefore computationally tractable. We also verified its prediction performance on a comprehensive, in-house subjective HDR video database with 90 sequences, and it was found to be better than some of the existing methods in terms of correlation with subjective scores (for both across sequence and per sequence cases). A software implementation of the proposed scheme is also made publicly available for free download and use. HighlightsThe paper presents one of the first objective method for high dynamic range video quality estimation.It is based on analysis of short term video segments taking into account human viewing behavior.The method described in the paper would be useful in scenarios where HDR video quality needs to be determined in an HDR video chain study.

Hdr-vqm

Recent advances in high dynamic range (HDR) capture and display technologies have attracted a lot of interest from scientific, professional, and artistic communities. As in any technology, the evaluation of HDR systems in terms of quality of experience is essential. Subjective evaluations are time consuming and expensive, and thus objective quality assessment tools are needed as well. In this paper, we report and analyze the results of an extensive benchmarking of objective quality metrics for HDR image quality assessment. In total, 35 objective metrics were benchmarked on a database of 20 HDR contents encoded with 3 compression algorithms at 4 bit rates, leading to a total of 240 compressed HDR images, using subjective quality scores as ground truth. Performance indexes were computed to assess the accuracy, monotonicity, and consistency of the metric estimation of subjective scores. Statistical analysis was performed on the performance indexes to discriminate small differences between metrics. Results demonstrated that metrics designed for HDR content, i.e., HDR-VDP-2 and HDR-VQM, are the most reliable predictors of perceived quality. Finally, our findings suggested that the performance of most full-reference metrics can be improved by considering non-linearities of the human visual system, while further efforts are necessary to improve performance of no-reference quality metrics for HDR content.

/pdf/benchmarking-of-objective-quality-metrics-for-hdr-image-1kh6q209da.pdf

Benchmarking of objective quality metrics for HDR image quality assessment

Standards play an important role in providing a common set of specifications and allowing inter-operability between devices and systems. Until recently, no standard for high-dynamic-range (HDR) image coding had been adopted by the market, and HDR imaging relies on proprietary and vendor-specific formats which are unsuitable for storage or exchange of such images. To resolve this situation, the JPEG Committee is developing a new coding standard called JPEG XT that is backward compatible to the popular JPEG compression, allowing it to be implemented using standard 8-bit JPEG coding hardware or software. In this paper, we present design principles and technical details of JPEG XT. It is based on a two-layer design, a base layer containing a low-dynamic-range image accessible to legacy implementations, and an extension layer providing the full dynamic range. The paper introduces three of currently defined profiles in JPEG XT, each constraining the common decoder architecture to a subset of allowable configurations. We assess the coding efficiency of each profile extensively through subjective assessments, using 24 naive subjects to evaluate 20 images, and objective evaluations, using 106 images with five different tone-mapping operators and at 100 different bit rates. The objective results (based on benchmarking with subjective scores) demonstrate that JPEG XT can encode HDR images at bit rates varying from 1.1 to 1.9 bit/pixel for estimated mean opinion score (MOS) values above 4.5 out of 5, which is considered as fully transparent in many applications. This corresponds to 23-times bitstream reduction compared to lossless OpenEXR PIZ compression.

/pdf/overview-and-evaluation-of-the-jpeg-xt-hdr-image-compression-naipxs5zd2.pdf

Overview and evaluation of the JPEG XT HDR image compression standard

Advances in computer graphics enable us to create digital images of astonishing complexity and realism. However, processing resources are still a limiting factor. Hence, many costly but desirable aspects of realism are often not accounted for, including global illumination, accurate depth of field and motion blur, spectral effects, etc. especially in real-time rendering. At the same time, there is a strong trend towards more pixels per display due to larger displays, higher pixel densities or larger fields of view. Further observable trends in current display technology include more bits per pixel high dynamic range, wider color gamut/fidelity, increasing refresh rates better motion depiction, and an increasing number of displayed views per pixel stereo, multi-view, all the way to holographic or lightfield displays. These developments cause significant unsolved technical challenges due to aspects such as limited compute power and bandwidth. Fortunately, the human visual system has certain limitations, which mean that providing the highest possible visual quality is not always necessary. In this report, we present the key research and models that exploit the limitations of perception to tackle visual quality and workload alike. Moreover, we present the open problems and promising future research targeting the question of how we can minimize the effort to compute and display only the necessary pixels while still offering a user full visual experience.

Perception-driven Accelerated Rendering

Due to the much larger luminance and contrast characteristics of high dynamic range (HDR) images, well-known objective quality metrics, widely used for the assessment of low dynamic range (LDR) content, cannot be directly applied to HDR images in order to predict their perceptual fidelity. To overcome this limitation, advanced fidelity metrics, such as the HDR-VDP, have been proposed to accurately predict visually significant differences. However, their complex calibration may make them difficult to use in practice. A simpler approach consists in computing arithmetic or structural fidelity metrics, such as PSNR and SSIM, on perceptually encoded luminance values but the performance of quality prediction in this case has not been clearly studied. In this paper, we aim at providing a better comprehension of the limits and the potentialities of this approach, by means of a subjective study. We compare the performance of HDR-VDP to that of PSNR and SSIM computed on perceptually encoded luminance values, when considering compressed HDR images. Our results show that these simpler metrics can be effectively employed to assess image fidelity for applications such as HDR image compression.

/pdf/performance-evaluation-of-objective-quality-metrics-for-hdr-50y9k48qfz.pdf

Performance evaluation of objective quality metrics for HDR image compression

The goal of this paper is to assess whether existing solutions to expand low dynamic range (LDR) still images to high dynamic range (HDR) displays can apply to moving pictures. To this end, we conduct a subjective quality assessment test on four state-of-the-art expansion operators, using high-quality input LDR content. Our results show that, perhaps surprisingly, no temporal artifacts, such as flickering due to global illumination changes, are introduced applying frame-based algorithms. In addition, they confirm previous studies in that operators as simple as linear expansion can yield acceptable (or even excellent) quality of the HDR video.

/pdf/dynamic-range-expansion-of-video-sequences-a-subjective-46hndubz6i.pdf

Dynamic range expansion of video sequences: A subjective quality assessment study

A core part of the state-of-the art high dynamic range (HDR) image and video compression methods is the tone mapping operation to convert the visible luminance range into the finite bit depths that can be supported by the current video codecs. These conversions are until now optimized to provide backward compatibility to the existing low dynamic range (LDR) displays. However, a direct application of these methods for the emerging HDR displays can result in a loss of details in the bright and dark regions of the HDR content. In this paper, we overcome this limitation by designing a tone mapping operation which handles the bright and dark regions separately. The proposed method first finds the optimal segmentation of the HDR image into two parts, namely dark and bright regions, and then designs the optimal tone mapping for each region in terms of the mean square error between the logarithm of the luminance values of the original and reconstructed HDR content (HDR-MSE). The results indicate the superiority of the proposed method over the state-of-the art HDR coding methods.

/pdf/segmentation-based-optimized-tone-mapping-for-high-dynamic-1fak7qao6a.pdf

Segmentation-based optimized tone mapping for high dynamic range image and video coding

A common paradigm to code high dynamic range (HDR) im-age/video content is based on tone-mapping HDR pictures to low dynamic range (LDR), in order to obtain backward compatibility and use existing coding tools, and then use inverse tone mapping at the decoder to predict the original HDR signal. Clearly, the choice of a proper tone mapping is essential in order to achieve good coding performance. The state-of-the-art to design the optimal tone mapping operator (TMO) minimizes the mean-square-error distortion between the original and the predicted HDR image. In this paper, we argue that this is suboptimal in rate-distortion sense, and we propose a more effective TMO design strategy that takes into account also the spatial complexity (which is a proxy for the bitrate) of the coded LDR image. Our results show that the proposed optimization approach enables to obtain substantial coding gain with respect to the minimum-MSE TMO.

/pdf/improved-tone-mapping-operator-for-hdr-coding-optimizing-the-51mx5prpnn.pdf

Paul Lauga

Papers

Performance evaluation of objective quality metrics for HDR image compression

Dynamic range expansion of video sequences: A subjective quality assessment study

Segmentation-based optimized tone mapping for high dynamic range image and video coding

Improved tone mapping operator for HDR coding optimizing the distortion/spatial complexity trade-off.

Spatio-temporal constrained tone mapping operator for HDR video compression