Showing papers on "Tone mapping published in 2014"

Creating cinematic wide gamut HDR-video for the evaluation of tone mapping operators and HDR-displays

Abstract: High quality video sequences are required for the evaluation of tone mapping operators and high dynamic range (HDR) displays. We provide scenic and documentary scenes with a dynamic range of up to 18 stops. The scenes are staged using professional film lighting, make-up and set design to enable the evaluation of image and material appearance. To address challenges for HDR-displays and temporal tone mapping operators, the sequences include highlights entering and leaving the image, brightness changing over time, high contrast skin tones, specular highlights and bright, saturated colors. HDR-capture is carried out using two cameras mounted on a mirror-rig. To achieve a cinematic depth of field, digital motion picture cameras with Super-35mm size sensors are used. We provide HDR-video sequences to serve as a common ground for the evaluation of temporal tone mapping operators and HDR-displays. They are available to the scientific community for further research.

Exposure fusion using boosting Laplacian pyramid.

Abstract: This paper proposes a new exposure fusion approach for producing a high quality image result from multiple exposure images. Based on the local weight and global weight by considering the exposure quality measurement between different exposure images, and the just noticeable distortion-based saliency weight, a novel hybrid exposure weight measurement is developed. This new hybrid weight is guided not only by a single image's exposure level but also by the relative exposure level between different exposure images. The core of the approach is our novel boosting Laplacian pyramid, which is based on the structure of boosting the detail and base signal, respectively, and the boosting process is guided by the proposed exposure weight. Our approach can effectively blend the multiple exposure images for static scenes while preserving both color appearance and texture structure. Our experimental results demonstrate that the proposed approach successfully produces visually pleasing exposure fusion images with better color appearance and more texture details than the existing exposure fusion techniques and tone mapping operators.

Generalized Equalization Model for Image Enhancement

Abstract: In this paper, we propose a generalized equalization model for image enhancement. Based on our analysis on the relationships between image histogram and contrast enhancement/white balancing, we first establish a generalized equalization model integrating contrast enhancement and white balancing into a unified framework of convex programming of image histogram. We show that many image enhancement tasks can be accomplished by the proposed model using different configurations of parameters. With two defining properties of histogram transform, namely contrast gain and nonlinearity, the model parameters for different enhancement applications can be optimized. We then derive an optimal image enhancement algorithm that theoretically achieves the best joint contrast enhancement and white balancing result with trading-off between contrast enhancement and tonal distortion. Subjective and objective experimental results show favorable performances of the proposed algorithm in applications of image enhancement, white balancing and tone correction. Computational complexity of the proposed method is also analyzed.

High-Quality Reverse Tone Mapping for a Wide Range of Exposures

Abstract: Reverse-tone-mapping operators (rTMOs) enhance low-dynamic-range images and videos for display on high-dynamic-range monitors. A common problem faced by previous rTMOs is the handling of under or overexposed content. Under such conditions, they may not be effective, and even cause loss and reversal of visible contrast. We present an rTMO based on cross-bilateral filtering that generates high-quality HDR images and videos for a wide range of exposures. Experiments performed using an objective image quality metric show that our approach is the only technique available that can gracefully enhance perceived details across a large range of image exposures.

Physiological inverse tone mapping based on retina response

Abstract: The mismatch between the Low Dynamic Range (LDR) content and the High Dynamic Range (HDR) display arouses the research on inverse tone mapping algorithms. In this paper, we present a physiological inverse tone mapping algorithm inspired by the property of the Human Visual System (HVS). It first imitates the retina response and deduce it to be local adaptive; then estimates local adaptation luminance at each point in the image; finally, the LDR image and local luminance are applied to the inversed local retina response to reconstruct the dynamic range of the original scene. The good performance and high-visual quality were validated by operating on 40 test images. Comparison results with several existing inverse tone mapping methods prove the conciseness and efficiency of the proposed algorithm.

Matching by Tone Mapping: Photometric Invariant Template Matching

Abstract: A fast pattern matching scheme termed matching by tone mapping (MTM) is introduced which allows matching under nonlinear tone mappings. We show that, when tone mapping is approximated by a piecewise constant/linear function, a fast computational scheme is possible requiring computational time similar to the fast implementation of normalized cross correlation (NCC). In fact, the MTM measure can be viewed as a generalization of the NCC for nonlinear mappings and actually reduces to NCC when mappings are restricted to be linear. We empirically show that the MTM is highly discriminative and robust to noise with comparable performance capability to that of the well performing mutual information, but on par with NCC in terms of computation time.

Visual-Salience-Based Tone Mapping for High Dynamic Range Images

Abstract: Visual saliency aims to predict the attentional gaze of observers viewing a scene, and it is thus highly demanded for tone mapping of high dynamic range (HDR) images. In this paper, novel saliency-aware weighting and edge-aware weighting are introduced for HDR images. They are incorporated into an existing guided image filter to form a perceptually guided image filter. The saliency-aware weighting and the proposed filter are applied to design a new local tone-mapping algorithm for HDR images such that both extreme light and shadow regions can be reproduced on conventional low dynamic range displays. In particular, the proposed filter is applied to decompose the luminance of the input HDR image into a base layer and a detail layer. The saliency-aware weighting is then adopted to design a saliency-aware global tone mapping for the compression of the base layer. The proposed filter preserves sharp edges in the base layer better than the existing guided filter. Halo artifacts are thus significantly reduced in the tone-mapped image. Moreover, the visual quality of the tone-mapped image, especially attention-salient regions, is improved by the saliency-aware weighting.

Temporally coherent local tone mapping of HDR video

Abstract: Recent subjective studies showed that current tone mapping operators either produce disturbing temporal artifacts, or are limited in their local contrast reproduction capability. We address both of these issues and present an HDR video tone mapping operator that can greatly reduce the input dynamic range, while at the same time preserving scene details without causing significant visual artifacts. To achieve this, we revisit the commonly used spatial base-detail layer decomposition and extend it to the temporal domain. We achieve high quality spatiotemporal edge-aware filtering efficiently by using a mathematically justified iterative approach that approximates a global solution. Comparison with the state-of-the-art, both qualitatively, and quantitatively through a controlled subjective experiment, clearly shows our method's advantages over previous work. We present local tone mapping results on challenging high resolution scenes with complex motion and varying illumination. We also demonstrate our method's capability of preserving scene details at user adjustable scales, and its advantages for low light video sequences with significant camera noise.

Image pre-processing for optimizing automated photogrammetry performances

Abstract: . The purpose of this paper is to analyze how optical pre-processing with polarizing filters and digital pre-processing with HDR imaging, may improve the automated 3D modeling pipeline based on SFM and Image Matching, with special emphasis on optically non-cooperative surfaces of shiny or dark materials. Because of the automatic detection of homologous points, the presence of highlights due to shiny materials, or nearly uniform dark patches produced by low reflectance materials, may produce erroneous matching involving wrong 3D point estimations, and consequently holes and topological errors on the mesh originated by the associated dense 3D cloud. This is due to the limited dynamic range of the 8 bit digital images that are matched each other for generating 3D data. The same 256 levels can be more usefully employed if the actual dynamic range is compressed, avoiding luminance clipping on the darker and lighter image areas. Such approach is here considered both using optical filtering and HDR processing with tone mapping, with experimental evaluation on different Cultural Heritage objects characterized by non-cooperative optical behavior. Three test images of each object have been captured from different positions, changing the shooting conditions (filter/no-filter) and the image processing (no processing/HDR processing), in order to have the same 3 camera orientations with different optical and digital pre-processing, and applying the same automated process to each photo set.

High dynamic range and tone mapping imaging techniques

Abstract: Algorithms for improving the performance of conventional tone mapping operators (TMO) by calculating both a contrast waste score and a contrast loss score for a first tone-mapped image produced by the TMO. The two contrast scores can be used to optimize the performance of the TMO by reducing noise and improving contrast. Algorithms for generating an HDR image by converting non-linear color space images into linear color space format, aligning the images to a reference, de-ghosting the aligned images if necessary, and merging the aligned (and potentially de-ghosted) images to create an HDR image. The merging can be performed with exposure fusion, HDR reconstruction, or other suitable techniques.

Methods and Apparatus for Tone Mapping High Dynamic Range Images

Abstract: Methods, apparatus, and computer-readable storage media for tone mapping High Dynamic Range (HDR) images An input HDR image is separated into luminance and color Luminance is processed to obtain a base layer and a detail layer The base layer is compressed according to a non-linear remapping function to reduce the dynamic range, and the detail layer is adjusted The layers are combined to generate output luminance, and the output luminance and color are combined to generate an output image A base layer compression technique may be used that analyzes the details and compresses the base layer accordingly to provide space at the top of the intensity scale where the details are displayed to thus generate output images that are visually better than images generated using conventional techniques User interface elements may be provided via which a user may control one or more parameters of the tone mapping method

Modeling Radiometric Uncertainty for Vision with Tone-Mapped Color Images.

Abstract: To produce images that are suitable for display, tone-mapping is widely used in digital cameras to map linear color measurements into narrow gamuts with limited dynamic range. This introduces non-linear distortion that must be undone, through a radiometric calibration process, before computer vision systems can analyze such photographs radiometrically. This paper considers the inherent uncertainty of undoing the effects of tone-mapping. We observe that this uncertainty varies substantially across color space, making some pixels more reliable than others. We introduce a model for this uncertainty and a method for fitting it to a given camera or imaging pipeline. Once fit, the model provides for each pixel in a tone-mapped digital photograph a probability distribution over linear scene colors that could have induced it. We demonstrate how these distributions can be useful for visual inference by incorporating them into estimation algorithms for a representative set of vision tasks.

Zonal brightness coherency for video tone mapping

Abstract: Tone Mapping Operators (TMOs) compress High Dynamic Range (HDR) contents to address Low Dynamic Range (LDR) displays. While many solutions have been designed over the last decade, only few of them can cope with video sequences. Indeed, these TMOs tone map each frame of a video sequence separately, which results in temporal incoherency. Two main types of temporal incoherency are usually considered: flickering artifacts and temporal brightness incoherency. While the reduction of flickering artifacts has been well studied, less work has been performed on brightness incoherency. In this paper, we propose a method that aims at preserving spatio-temporal brightness coherency when tone mapping video sequences. Our technique computes HDR video zones which are constant throughout a sequence, based on the luminance of each pixel. Our method aims at preserving the brightness coherency between the brightest zone of the video and each other zone. This technique adapts to any TMO and results show that it preserves well spatio-temporal brightness coherency. We validate our method using a subjective evaluation. In addition, unlike local TMOs, our method, when applied to still images, is capable of ensuring spatial brightness coherency. Finally, it also preserves video fade effects commonly used in post-production.

Tone mapping based HDR compression: Does it affect visual experience?

Abstract: High Dynamic Range (HDR) content is visually more appealing since it can represent the real luminance of the scene. However, on the downside, this means that a large amount of data needs to be handled both during storage and processing. The other problem is that HDR content cannot be displayed on the conventional display devices due to their limited dynamic range. To overcome these two problems, dynamic range compression (or range reduction) is often used and this is accomplished by tone mapping operators (TMOs). As result of tone mapping, the HDR content is not only fit to be displayed on a regular display device but also compressed. However from an artistic intention point of view, TMOs are not necessarily transparent and might induce different viewing behavior. It is generally accepted that TMOs reduce visual quality and there have been a number of studies reported in literature which examine the impact of tone mapping from the view point of perceptual quality. In contrast to this, it is largely unclear if tone mapping will induce changes in visual attention (VA) as well and whether these are significant enough to be accounted for in HDR content processing. To our knowledge, no systematic study exists which sheds light on this issue. Given that VA is a crucial visual perception mechanism which affects the way we perceive visual signals, it is important to study the effect of tone mapping on VA deployment. Towards this goal, this paper investigates and quantifies how TMOs modify VA. Comprehensive subjective tests in the form of eye-tracking experiments have been conducted on several HDR content and using a large number of TMOs. Further non-parametric statistical analysis has been carried out to ascertain the statistical significance of the results obtained. Our studies suggest that TMOs can indeed modify human attention and fixation behavior. Based on this we believe that VA needs consideration for evaluating the overall perceptual impact of TMOs on HDR content. As mentioned, since the existing studies so far have only considered the quality or esthetic appeal angle, this study brings in a new perspective regarding the importance of VA in HDR content processing for visualization on LDR displays.

High dynamic range image tone mapping by optimizing tone mapped image quality index

Abstract: An active research topic in recent years is to design tone mapping operators (TMOs) that convert high dynamic range (H-DR) to low dynamic range (LDR) images, so that HDR images can be visualized on standard displays. Nevertheless, most existing work has been done in the absence of a well-established and subject-validated image quality assessment (IQA) model, without which fair comparisons and further improvement are difficult. Recently, a tone mapped image quality index (TMQI) was proposed, which has shown to have good correlation with subjective evaluations of tone mapped images. Here we propose a substantially different approach to design TMO, where instead of using any pre-defined systematic computational structure (such as image transformation or contrast/edge enhancement) for tone mapping, we navigate in the space of all images, searching for the image that optimizes TMQI. The navigation involves an iterative process that alternately improves the structural fidelity and statistical naturalness of the resulting image, which are the two fundamental building blocks in TMQI. Experiments demonstrate the superior performance of the proposed method.

Methods and systems for inverse tone mapping

Abstract: Novel methods and systems for inverse tone mapping are disclosed. A scene can be analyzed to obtain highlight detection from bright light sources and specular reflections. An inverse tone mapping curve can be calculated based on the lower dynamic range and higher dynamic range displays. Multi-scale filtering can be applied to reduce noise or artifacts.

Details preservation inspired blind quality metric of tone mapping methods

Abstract: High dynamic range (HDR) images are extremely meaningful, especially in the space and medical fields. For visualization of HDR images on standard low dynamic range (LDR) display devices, how to convert HDR to LDR images naturally becomes a valuable issue, which has aroused a variety of tone-mapping operators (TMOs). To compare different LDR images created by distinct TMOs, researchers have recently provided a subject-rated tone-mapped image database, and then developed a full-reference objective tone-mapped image quality index (TMQI) based on the measurement of multi-scale signal fidelity and statistical naturalness. Instead, the basic property of HDR images about details preservation is studied in this paper. With it, a natural inference is that higher-quality tone-mapped images are capable of displaying much more details. We therefore propose a blind quality metric by estimating the amount of details in images generated by darkening/brightening an original tone-mapped images. Experimental results on the above tone-mapped image database confirm that the proposed method, despite of no reference, is robust and statistically superior to the currently optimal full-reference TMQI algorithm, and remarkably outperforms state-of-the-art no-reference IQA metrics.

Evaluation of HDR video tone mapping for mobile devices

Abstract: Tone mapping operators (TMOs) allow the visual appearance of High Dynamic Range (HDR) video to be reproduced on Low Dynamic Range (LDR) displays. While several studies have been made to compare the performance of different TMOs on standard displays, there is only one preliminary study that takes into account the characteristics of Small Screen Devices (SSDs) which are common on mobile platforms that are growing rapidly in popularity. Due to the unique characteristics of mobile devices, the variety of viewing conditions where they are used, and the fact that they are becoming so widespread, it is important to identify what is the best method to deliver HDR video content to these devices. This paper presents an evaluation of HDR video on mobile devices using an HDR display as reference. Six state-of-the-art HDR video tone mappers are evaluated by conducting a psychophysical experiment where participants were asked to rank the tone mappers applied to different HDR video footage. A comparison was made between tone mapped HDR video footage shown on a tablet and an LCD display compared with the same HDR video footage shown simultaneously on an HDR display. This study shows there is a subtle but statistically significant difference in the preference of TMOs between SSD and LDR even though the overall ordering of TMOs is the same across both displays.

Methods for improving the tone mapping for backward compatible high dynamic range image and video coding

Abstract: Backward compatibility for high dynamic range image and video compression forms one of the essential requirements in the transition phase from low dynamic range (LDR) displays to high dynamic range (HDR) displays. In a recent work [1], the problems of tone mapping and HDR video coding are originally fused together in the same mathematical framework, and an optimized solution for tone mapping is achieved in terms of the mean square error (MSE) of the logarithm of luminance values. In this paper, we improve this pioneer study in three aspects by considering its three shortcomings. First, the proposed method [1] works over the logarithms of luminance values which are not uniform with respect to Human Visual System (HVS) sensitivity. We propose to use the perceptually uniform luminance values as an alternative for the optimization of tone mapping curve. Second, the proposed method [1] does not take the quality of the resulting tone mapped images into account during the formulation in contrary to the main goal of tone mapping research. We include the LDR image quality as a constraint to the optimization problem and develop a generic methodology to compromise the trade-off between HDR and LDR image qualities for coding. Third, the proposed method [1] simply applies a low-pass filter to the generated tone curves for video frames to avoid flickering during the adaptation of the method to the video. We instead include an HVS based flickering constraint to the optimization and derive a methodology to compromise the trade-off between the rate-distortion performance and flickering distortion. The superiority of the proposed methodologies is verified with experiments on HDR images and video sequences.

Weighted Least Squares Based Detail Enhanced Exposure Fusion

Abstract: Many recent computational photography techniques play a significant role to avoid limitation of standard digital cameras to handle wide dynamic range of the real-world scenes, containing brightly and poorly illuminated areas. In many of these techniques, it is often desirable to fuse details from images captured at different exposure settings, while avoiding visual artifacts. In this paper we propose a novel technique for exposure fusion in which Weighted Least Squares (WLS) optimization framework is utilized for weight map refinement. Computationally simple texture features (i.e., detail layer extracted with the help of edge preserving filter) and color saturation measure are preferred for quickly generating weight maps to control the contribution from an input set of multiexposure images. Instead of employing intermediate High Dynamic Range (HDR) reconstruction and tone mapping steps, well-exposed fused image is generated for displaying on conventional display devices. A further advantage of the present technique is that it is well suited for multifocus image fusion. Simulation results are compared with a number of existing single resolution and multiresolution techniques to show the benefits of the proposed scheme for variety of cases.

An integer tone mapping operation for HDR images in OpenEXR with denormalized numbers

Abstract: We propose an integer tone mapping operator (TMO) for high dynamic range (HDR) images expressed in a floating-point data format. Two purposes are achieved by the proposed TMO. The first purpose is to implement a TMO with less memory space. The second purpose is to give an important step to realize a fixed-point TMO. The proposed TMO is available for HDR images in the OpenEXR format. The OpenEXR format has two numerical representations (the normalized number and the denormalized number) which are not in other HDR formats such as RGBE. These two numerical representations cause a problem in applying an integer TMO. The proposed method enables us to avoid the problem by using the intermediate format. Moreover, the exponent part and the mantissa part are processed separately as two integer numbers. As a result, an integer TMO with less numerical range is achieved by our method. The experimental results show that the proposed method can generate high-quality low dynamic range (LDR) images with less memory space.

A novel approach for detail-enhanced exposure fusion using guided filter.

Abstract: In this paper we propose a novel detail-enhancing exposure fusion approach using nonlinear translation-variant filter (NTF). With the captured Standard Dynamic Range (SDR) images under different exposure settings, first the fine details are extracted based on guided filter. Next, the base layers (i.e., images obtained from NTF) across all input images are fused using multiresolution pyramid. Exposure, contrast, and saturation measures are considered to generate a mask that guides the fusion process of the base layers. Finally, the fused base layer is combined with the extracted fine details to obtain detail-enhanced fused image. The goal is to preserve details in both very dark and extremely bright regions without High Dynamic Range Image (HDRI) representation and tone mapping step. Moreover, we have demonstrated that the proposed method is also suitable for the multifocus image fusion without introducing artifacts.

Survey of Temporal Brightness Artifacts in Video Tone Mapping

Abstract: High Dynamic Range video acquisition and display are now achievable thanks to recent advances in the HDR field Tone Mapping Operators convert HDR images to Low Dynamic Range ones However, most of these operators were designed for images and do not take into account the temporal consistency inherent in a video sequence Consequently, the appearance of temporal artifacts impairs the quality of the tone mapped video content We propose, in this article, to classify temporal artifacts into different types: flickering, temporal contrast and object incoherency Then, we describe some techniques designed to solve these issues and the temporal artifacts that they generate Finally, we conclude by differentiating the interactive and automatic tone mapping requirements

Performance Optimization and FPGA Implementation of Real-Time Tone Mapping

Abstract: This brief analyzes the performance of the hardware-based tone mapping operators for compression of high dynamic range images. The bottlenecks of a tone mapping system are determined and a high-performance field programmable gate array (FPGA) implementation of an operator is introduced. The operator utilizes polynomial mapping technique, adaptive to the pixel values; hence preserving high contrast areas. The technique is further optimized for the presented resource-efficient FPGA implementation. We show that the timing optimization does not reduce the image quality, by obtaining high peak signal-to-noise ratio of the resulting images. The timing comparison to the similar implementations shows 2.5 times increase in the achieved throughput, irrespective of the hardware platform.

System and method for lens shading compensation

Abstract: A method for performing lens shading compensation is provided. The method includes: receiving at least two series of source images of different exposure settings; performing first lens shading compensation on the at least two series of source images respectively; analyzing luminance distribution of the at least two series of compensated source images; composing a series of HDR images from the at least two series of compensated source images according to the luminance distribution; performing second lens shading compensation on the series of HDR image; and performing tone mapping on the series of compensated HDR images. A system for performing lens shading compensation is also provided.

Content adaptive guided image filtering

Abstract: Local filtering based edge-preserving smoothing technique could suffer from halo artifacts when it is applied for image enhancement. In this paper, an adaptive guided image filter is proposed by incorporating edge aware weighting which is derived from normalized local variance of a guidance image into an existing guided filter. It is shown that the proposed filter preserves sharp edges better than the existing guided filter. With the observation, it is applied for tone mapping of high dynamic range images and fusion of differently exposed images. Experimental results show that the resultant tone mapping and exposure fusion algorithms can prevent halo artifacts from appearing in the final images.