Showing papers on "Tone mapping published in 2015"

Gradient Domain Guided Image Filtering

Abstract: Guided image filter (GIF) is a well-known local filter for its edge-preserving property and low computational complexity. Unfortunately, the GIF may suffer from halo artifacts, because the local linear model used in the GIF cannot represent the image well near some edges. In this paper, a gradient domain GIF is proposed by incorporating an explicit first-order edge-aware constraint. The edge-aware constraint makes edges be preserved better. To illustrate the efficiency of the proposed filter, the proposed gradient domain GIF is applied for single-image detail enhancement, tone mapping of high dynamic range images and image saliency detection. Both theoretical analysis and experimental results prove that the proposed gradient domain GIF can produce better resultant images, especially near the edges, where halos appear in the original GIF.

Local Laplacian filters: edge-aware image processing with a Laplacian pyramid

Abstract: The Laplacian pyramid is ubiquitous for decomposing images into multiple scales and is widely used for image analysis. However, because it is constructed with spatially invariant Gaussian kernels, the Laplacian pyramid is widely believed to be ill-suited for representing edges, as well as for edge-aware operations such as edge-preserving smoothing and tone mapping. To tackle these tasks, a wealth of alternative techniques and representations have been proposed, for example, anisotropic diffusion, neighborhood filtering, and specialized wavelet bases. While these methods have demonstrated successful results, they come at the price of additional complexity, often accompanied by higher computational cost or the need to postprocess the generated results. In this paper, we show state-of-the-art edge-aware processing using standard Laplacian pyramids. We characterize edges with a simple threshold on pixel values that allow us to differentiate large-scale edges from small-scale details. Building upon this result, we propose a set of image filters to achieve edge-preserving smoothing, detail enhancement, tone mapping, and inverse tone mapping. The advantage of our approach is its simplicity and flexibility, relying only on simple point-wise nonlinearities and small Gaussian convolutions; no optimization or postprocessing is required. As we demonstrate, our method produces consistently high-quality results, without degrading edges or introducing halos.

High Dynamic Range Image Compression by Optimizing Tone Mapped Image Quality Index

Abstract: Tone mapping operators (TMOs) aim to compress high dynamic range (HDR) images to low dynamic range (LDR) ones so as to visualize HDR images on standard displays. Most existing TMOs were demonstrated on specific examples without being thoroughly evaluated using well-designed and subject-validated image quality assessment models. A recently proposed tone mapped image quality index (TMQI) made one of the first attempts on objective quality assessment of tone mapped images. Here, we propose a substantially different approach to design TMO. Instead of using any predefined systematic computational structure for tone mapping (such as analytic image transformations and/or explicit contrast/edge enhancement), we directly navigate in the space of all images, searching for the image that optimizes an improved TMQI. In particular, we first improve the two building blocks in TMQI—structural fidelity and statistical naturalness components—leading to a TMQI-II metric. We then propose an iterative algorithm that alternatively improves the structural fidelity and statistical naturalness of the resulting image. Numerical and subjective experiments demonstrate that the proposed algorithm consistently produces better quality tone mapped images even when the initial images of the iteration are created by the most competitive TMOs. Meanwhile, these results also validate the superiority of TMQI-II over TMQI. 1 1 Partial preliminary results of this work were presented at ICASSP 2013 and ICME 2014.

Pseudo-Multiple-Exposure-Based Tone Fusion With Local Region Adjustment

Abstract: New generations of display technologies provide a significantly improved dynamic range compared to conventional display devices. Inverse tone mapping methods have been proposed to convert low dynamic range (LDR) images to HDR ones, and several of them require multiple exposure LDR images of the same scene as inputs. However, the vast majority of LDR images and videos available have only one single exposure. In this paper, we propose a region-based enhancement of the pseudo-exposures to generate an HDR image. First, we present an exposure dependent $S$ curve to convert one LDR image to the pseudo-multiple-exposures. Only certain regions of the pseudo-exposures contain noticeable detail information. We propose a region-based enhancement on the pseudo-exposures to boost details in the most distinct region. Thereby the region-enhanced pseudo-exposures are fused into an HDR image. The fused image thus enhances details in the bright region of the dark image and the dark region of the bright image. Compared with other inverse tone mapped methods, our method generates lower total contrast error measured under the dynamic range independent image quality assessment method in .

FSITM: A Feature Similarity Index For Tone-Mapped Images

Abstract: In this work, based on the local phase information of images, an objective index, called the feature similarity index for tone-mapped images (FSITM), is proposed. To evaluate a tone mapping operator (TMO), the proposed index compares the locally weighted mean phase angle map of an original high dynamic range (HDR) to that of its associated tone-mapped image calculated using the output of the TMO method. In experiments on two standard databases, it is shown that the proposed FSITM method outperforms the state-of-the-art index, the tone mapped quality index (TMQI). In addition, a higher performance is obtained by combining the FSITM and TMQI indices.

Real-time noise-aware tone mapping

Abstract: Real-time high quality video tone mapping is needed for many applications, such as digital viewfinders in cameras, display algorithms which adapt to ambient light, in-camera processing, rendering engines for video games and video post-processing. We propose a viable solution for these applications by designing a video tone-mapping operator that controls the visibility of the noise, adapts to display and viewing environment, minimizes contrast distortions, preserves or enhances image details, and can be run in real-time on an incoming sequence without any preprocessing. To our knowledge, no existing solution offers all these features. Our novel contributions are: a fast procedure for computing local display-adaptive tone-curves which minimize contrast distortions, a fast method for detail enhancement free from ringing artifacts, and an integrated video tone-mapping solution combining all the above features.

High Dynamic Range Imaging

Abstract: High dynamic range (HDR) images and video contain pixels, which can represent much greater range of colors and brightness levels than that offered by existing, standard dynamic range images. Such “better pixels” greatly improve the overall quality of visual content, making it appear much more realistic and appealing to the audience. HDR is one of the key technologies of the future imaging pipeline, which will change the way the digital visual content is represented and manipulated. This article offers a broad review of the HDR methods and technologies with an introduction to fundamental concepts behind the perception of HDR imagery. It serves as both an introduction to the subject and a review of the current state of the art in HDR imaging. It covers the topics related to capture of HDR content with cameras and its generation with computer graphics methods; encoding and compression of HDR images and video; tone mapping for displaying HDR content on standard dynamic range displays; inverse tone mapping for upscaling legacy content for presentation on HDR displays; the display technologies offering HDR range; and finally image and video quality metrics suitable for HDR content. Keywords: high dynamic range imaging; tone mapping

Display device and method of controlling the same

Abstract: A display device, including a content receiving unit configured to receive a high dynamic range image, an image processing unit configured to detect a first region whose luminance value is equal to or greater than a reference luminance value within the high dynamic range image and perform tone mapping on an image of the first region based on feature information of the image of the first region, and a display unit configured to display a low dynamic range image on which the tone mapping is performed.

Robust Match Fusion Using Optimization

Abstract: In this paper, we present a novel patch-based match and fusion algorithm by taking account of moving scene in a multiple exposure image sequence using optimization. A uniform iterative approach is developed to match and find the corresponding patches in different exposure images, which are then fused in each iteration. Our approach does not need to align the input multiple exposure images before the fusion process. Considering that the pixel values are affected by various exposure time, we design a new patch-based energy function that will be optimized to improve the matching accuracy. An efficient patch-based exposure fusion approach using the random walker algorithm is developed to preserve the moving objects from the input multiple exposure images. To the best of our knowledge, our algorithm is the first patch-based exposure fusion work to preserve the moving objects of dynamic scenes that does not need the registration process of different exposure images. Experimental results of moving scenes demonstrate that our algorithm achieves visually pleasing fusion results without ghosting artifacts, while the results produced by the state-of-the-art exposure fusion and tone mapping algorithms exhibit different levels of ghosting artifacts.

A novel approach for denoising and enhancement of extremely low-light video

Abstract: In this paper, a novel approach for noise reduction and enhancement of extremely low-light video is proposed. For noise removal, a motion adaptive temporal filtering based on a Kalman structured updating is presented. Dynamic range of denoised video is increased by adjustment of RGB histograms using Gamma correction with adaptive clipping thresholds. Finally, residual noise is removed using a nonlocal means (NLM) denoising filter. The proposed method works directly on the color filter array (CFA) raw video for achieving low memory consumption1.

Methods and apparatuses for encoding an HDR images, and methods and apparatuses for use of such encoded images

Abstract: A method of encoding a high dynamic range image (M_HDR), comprising the steps of: - converting the high dynamic range image into a low luminance dynamic range image (LDR_o) by applying: a) normalization of the image of high dynamic range at a luma axis scale that is [0, 1] giving a high normalized dynamic range image with normalized colors that have normalized luminance (Yn_HDR), b) calculate a gamma function on normalized luminance giving converted luminance with gamma (xg), c) apply a first tone mapping that gives lumas (v) that is defined as ** Formula **, with RHO having a predetermined value, and d) apply a monotonously increasing arbitrary tone mapping function that maps the lumas with output lumas (Yn_LDR) of the lower dynamic range image (LDR_o); and - emit, in an image signal (S_im), an encoding of the pixel colors of the lower luminance dynamic range image (LDR_o), and - emit, in the image signal (S_im), values encoding the function forms of previous color conversions bad as metadata, or values for their inverse functions, metadata that allow a receiver to reconstruct a reconstructed high dynamic range image (Rec_HDR) from the low luminance dynamic range image (LDR_o ), where RHO or a value that is a function of RHO is issued in the metadata.

A Brief Survey of Recent Edge-Preserving Smoothing Algorithms on Digital Images.

Abstract: Edge preserving filters preserve the edges and its information while blurring an image In other words they are used to smooth an image, while reducing the edge blurring effects across the edge like halos, phantom etc They are nonlinear in nature Examples are bilateral filter, anisotropic diffusion filter, guided filter, trilateral filter etc Hence these family of filters are very useful in reducing the noise in an image making it very demanding in computer vision and computational photography applications like denoising, video abstraction, demosaicing, optical-flow estimation, stereo matching, tone mapping, style transfer, relighting etc This paper provides a concrete introduction to edge preserving filters starting from the heat diffusion equation in olden to recent eras, an overview of its numerous applications, as well as mathematical analysis, various efficient and optimized ways of implementation and their interrelationships, keeping focus on preserving the boundaries, spikes and canyons in presence of noise Furthermore it provides a realistic notion for efficient implementation with a research scope for hardware realization for further acceleration

A 151 dB High Dynamic Range CMOS Image Sensor Chip Architecture With Tone Mapping Compression Embedded In-Pixel

Abstract:  Abstract—This paper presents a high dynamic range CMOS image sensor that implements an in-pixel content-aware adaptive global tone mapping algorithm during image capture operation. The histogram of the previous frame of an auxiliary image, which contains time stamp information, is employed as an estimation of the probability of illuminations impinging pixels at the present frame. The compression function of illuminations, namely Tone Mapping Curve (TMC), is calculated using this histogram. A QCIF resolution proof-of-concept prototype has been fabricated using a 0.35µm opto-flavored standard technology. The sensor is capable of mapping scenes with a maximum intra-frame dynamic range of 151dB (25-bits/pixel in linear representation) by compressing them to only 7-bits/pixel, while keeping visual quality in details and contrast. The in-pixel on-the-fly fully- parallel tone mapping achieves high frame rate allowing real- time HDR video (120dB@30fps).

A model of local adaptation

Abstract: The visual system constantly adapts to different luminance levels when viewing natural scenes. The state of visual adaptation is the key parameter in many visual models. While the time-course of such adaptation is well understood, there is little known about the spatial pooling that drives the adaptation signal. In this work we propose a new empirical model of local adaptation, that predicts how the adaptation signal is integrated in the retina. The model is based on psychophysical measurements on a high dynamic range (HDR) display. We employ a novel approach to model discovery, in which the experimental stimuli are optimized to find the most predictive model. The model can be used to predict the steady state of adaptation, but also conservative estimates of the visibility (detection) thresholds in complex images. We demonstrate the utility of the model in several applications, such as perceptual error bounds for physically based rendering, determining the backlight resolution for HDR displays, measuring the maximum visible dynamic range in natural scenes, simulation of afterimages, and gaze-dependent tone mapping.

Evaluation of Feature Detection in HDR Based Imaging Under Changes in Illumination Conditions

Abstract: High dynamic range (HDR) imaging enables to capture details in both dark and very bright regions of a scene, and is therefore supposed to provide higher robustness to illumination changes than conventional low dynamic range (LDR) imaging in tasks such as visual features extraction. However, it is not clear how much this gain is, and which are the best modalities of using HDR to obtain it. In this paper we evaluate the first block of the visual feature extraction pipeline, i.e., keypoint detection, using both LDR and different HDR-based modalities, when significant illumination changes are present in the scene. To this end, we captured a dataset with two scenes and a wide range of illumination conditions. On these images, we measure how the repeatability of either corner or blob interest points is affected with different LDR/HDR approaches. Our observations confirm the potential of HDR over conventional LDR acquisition. Moreover, extracting features directly from HDR pixel values is more effective than first tonemapping and then extracting features, provided that HDR luminance information is previously encoded to perceptually linear values.

Backward-compatible video capture and distribution

Abstract: Video processing techniques and pipelines that support capture, distribution, and display of high dynamic range (HDR) image data to both HDR-enabled display devices and display devices that do not support HDR imaging. A sensor pipeline may generate standard dynamic range (SDR) data from HDR data captured by a sensor using tone mapping, for example local tone mapping. Information used to generate the SDR data may be provided to a display pipeline as metadata with the generated SDR data. If a target display does not support HDR imaging, the SDR data may be directly rendered by the display pipeline. If the target display does support HDR imaging, then an inverse mapping technique may be applied to the SDR data according to the metadata to render HDR data for display. Information used in performing color gamut mapping may also be provided in the metadata and used to recover clipped colors for display.

Video tone mapping for converting high dynamic range (hdr) content to standard dynamic range (sdr) content

Abstract: Techniques are described for a method, an apparatus, a system and at least one machine readable medium to generate standard dynamic range videos from high dynamic range videos, the method comprising the steps of: applying an inverse gamma correction; applying a matrix multiplication that converts a color space; stretching a luminance range based at least in part on one or more stretching factors; and applying a forward gamma correction.

Optimal exposure compression for high dynamic range content

Abstract: High dynamic range (HDR) imaging has become one of the foremost imaging methods capable of capturing and displaying the full range of lighting perceived by the human visual system in the real world. A number of HDR compression methods for both images and video have been developed to handle HDR data, but none of them has yet been adopted as the method of choice. In particular, the backwards-compatible methods that always maintain a stream/image that allow part of the content to be viewed on conventional displays make use of tone mapping operators which were developed to view HDR images on traditional displays. There are a large number of tone mappers, none of which is considered the best as the images produced could be deemed subjective. This work presents an alternative to tone mapping-based HDR content compression by identifying a single exposure that can reproduce the most information from the original HDR image. This single exposure can be adapted to fit within the bit depth of any traditional encoder. Any additional information that may be lost is stored as a residual. Results demonstrate quality is maintained as well, and better, than other traditional methods. Furthermore, the presented method is backwards-compatible, straightforward to implement, fast and does not require choosing tone mappers or settings.

High dynamic range imaging pipeline on the GPU

Abstract: Use of high dynamic range (HDR) images and video in image processing and computer graphics applications is rapidly gaining popularity. However, creating and displaying high resolution HDR content on CPUs is a time consuming task. Although some previous work focused on real-time tone mapping, implementation of a full HDR imaging (HDRI) pipeline on the GPU has not been detailed. In this article we aim to fill this gap by providing a detailed description of how the HDRI pipeline, from HDR image assembly to tone mapping, can be implemented exclusively on the GPU. We also explain the trade-offs that need to be made for improving efficiency and show timing comparisons for CPU versus GPU implementations of the HDRI pipeline.

DaTuM: dynamic tone mapping technique for OLED display power saving based on video classification

Abstract: The adoption of the latest OLED (organic light emitting diode) technology does not change the fact that screen is still one of the most energy-consuming modules in modern smartphones. In this work, we found that video streams from the same video category share many common power consumption features on OLED screens. Therefore, we are able to build a Hidden Markov Model (HMM) classifier to categorize videos based on OLED screen power characteristics. Using this HMM classifier, we propose a video classification based dynamic tone mapping (DTM) scheme, namely, DaTuM, to remap output color range and minimize the power-hungry color compositions on OLED screens for power saving. Experiment shows that DaTuM scheme averagely reduces OLED screen power by 17.8% with minimum display quality degradation. Compared to DTM scheme based on official category info provided by the video sources and one state-of-the-art scheme, DaTuM substantially enhances OLED screens' power efficiency and display quality controllability.

High dynamic range tone mapping

Abstract: Embodiments herein disclose a tone mapping system that generates a base layer by filtering a frame in an HDR video. The tone mapping system then generates a tone curve using a tone mapping parameter derived from the base layer—e.g., a mean or maximum luminance value. Once the tone curve parameter is identified, the system performs temporal filtering to smooth out inconsistencies between the current value of the tone curve parameter and the values of tone curve parameter of at least one previous frame in the HDR video. The tone mapping system applies the temporally filtered tone curve to the base layer to generate a temporally coherent layer. The temporally coherent base layer can be combined with a detail layer derived from the HDR video to generate a frame of a tone mapped video.

Method and apparatus for encoding and decoding hdr images

Abstract: To encode High Dynamic Range (HDR) images, the HDR images can be converted to Low Dynamic Range (LDR) images through tone mapping operation, and the LDR images can be encoded with an LDR encoder. The present principles formulates a rate distortion minimization problem when designing the tone mapping curve. In particular, the tone mapping curve is formulated as a function of the probability distribution function of the HDR images to be encoded and a Lagrangian multiplier that depends on encoding parameters. At the decoder, based on the parameters indicative of the tone mapping function, an inverse tone mapping function can be derived to reconstruct HDR images from decoded LDR images.

Enhanced tone mapper for high dynamic range images and video

Abstract: Various embodiments provide tone mapping of images and video from one dynamic range to an available dynamic range of a display device while preserving or enhancing image details According to one embodiment, an enhanced tone mapping module is configured to decrease a luminance of an image or video from a high dynamic range to a standard dynamic range Conversely, according to one embodiment, an enhanced inverse tone mapper to increase a luminance of an image or video from a standard dynamic range to a high dynamic range

Methods and apparatuses for encoding hdr images

Abstract: To enable future video transmission of a large range of HDR videos, and render them on displays of variable dynamic range or peak brightness capability, we describe an encoder (301) for encoding an input high dynamic range video set of images (Im_5000), having pixel colors with luminances lower than a first maximum luminance (L_max_M), into an encoded high dynamic range video (Im_2000), being a high dynamic range image, i.e. with a maximum luminance for displaying on a display with a corresponding peak brightness of at least 900 nit, being characterized in that the encoding definition allows encoding pixel color luminances up to a second maximum luminance (L_max_C), which is equal to or less than 50% of the first maximum luminance, the encoder comprising: a re-grading unit (320) arranged to allow a content creator to specify at least a tone mapping function (F_2Tu, 601) for color mapping the encoded high dynamic range video (Im_2000) of lower second maximum luminance (L_max_C) to a HDR video reconstruction (Im_5000*) of the high dynamic range video (Im_5000); and a formatter, arranged to write into a video signal (S_im) the encoded high dynamic range video (Im_2000) and as metadata the at least a tone mapping function (F_2Tu, 601), and related encoder and decoder embodiments and methods, and transmission technical components and technologies.

Hdr Imaging for Feature Detection on Detailed Architectural Scenes

Abstract: 3D reconstruction relies on accurate detection, extraction, description and matching of image features This is even truer for complex architectural scenes that pose needs for 3D models of high quality, without any loss of detail in geometry or color Illumination conditions influence the radiometric quality of images, as standard sensors cannot depict properly a wide range of intensities in the same scene Indeed, overexposed or underexposed pixels cause irreplaceable information loss and degrade digital representation Images taken under extreme lighting environments may be thus prohibitive for feature detection/extraction and consequently for matching and 3D reconstruction High Dynamic Range (HDR) images could be helpful for these operators because they broaden the limits of illumination range that Standard or Low Dynamic Range (SDR/LDR) images can capture and increase in this way the amount of details contained in the image Experimental results of this study prove this assumption as they examine state of the art feature detectors applied both on standard dynamic range and HDR images

Saliency weighted quality assessment of tone-mapped images

Abstract: Different Tone-Mapping operators (TMOs) produce different Low Dynamic Range (LDR) images based on a single High Dynamic Range (HDR) image. The Tone-Mapped image Quality Index (TMQI) algorithm provides a quantitative means of assessing the quality of resultant LDR images. In this paper we test the hypothesis that TMQI predictions of human image quality can be further aligned with human judgement of image quality in considering visual attention, or regions that humans are predicted to fixate within a scene. We propose a modified version of the TMQI algorithm, a Saliency weighted Tone-Mapped Quality Index (STMQI) which demonstrates higher correlation with subjective ranking scores than the standard TMQI metric.

A critical look to some contrast enhancement evaluation measures

Abstract: Contrast enhancement is one of the most studied problems in image processing. A plethora of approaches has been proposed in the literature for image enhancement since the pioneer work of Kovasznay and Joseph in 1955 [1] and the famous contribution of Gabor in 1965 on image deblurring [2]. However, very few works have been dedicated to contrast enhancement evaluation. This is mainly due to the fact that image enhancement is primarily related to subjective aspects of human perceptual vision. The intent of this paper is to provide a first critical review to some contrast enhancement evaluation measures and propose a new one. An objective comparison of these measures on various color real images processed by some neighborhood based methods is provided. This work is considered as a first step towards the development of a unifying framework for image enhancement evaluation. This could be also used to control the side effect that may result from any image enhancement such as contrast enhancement, denoising, tone mapping and other similar image processing tools.

Evaluation of Tone-Mapping Operators for HDR Video Under Different Ambient Luminance Levels

Abstract: Since high dynamic range HDR displays are not yet widely available, there is still a need to perform a dynamic range reduction of HDR content to reproduce it properly on standard dynamic range SDR displays. The most common techniques for performing this reduction are termed tone-mapping operators TMOs. Although mobile devices are becoming widespread, methods for displaying HDR content on these SDR screens are still very much in their infancy. While several studies have been conducted to evaluate TMOs, few have been done with a goal of testing small screen displays SSDs, common on mobile devices. This paper presents an evaluation of six state-of-the-art HDR video TMOs. The experiments considered three different levels of ambient luminance under which 180 participants were asked to rank the TMOs for seven tone-mapped HDR video sequences. A comparison was conducted between tone-mapped HDR video footage shown on an SSD and on a large screen SDR display using an HDR display as reference. The results show that there are differences between the performance of the TMOs under different ambient lighting levels and the TMOs that perform well on traditional large screen displays also perform well on SSDs at the same given luminance level.