Showing papers on "Tone mapping published in 2007"

Joint bilateral upsampling

Abstract: Image analysis and enhancement tasks such as tone mapping, colorization, stereo depth, and photomontage, often require computing a solution (e.g., for exposure, chromaticity, disparity, labels) over the pixel grid. Computational and memory costs often require that a smaller solution be run over a downsampled image. Although general purpose upsampling methods can be used to interpolate the low resolution solution to the full resolution, these methods generally assume a smoothness prior for the interpolation. We demonstrate that in cases, such as those above, the available high resolution input image may be leveraged as a prior in the context of a joint bilateral upsampling procedure to produce a better high resolution solution. We show results for each of the applications above and compare them to traditional upsampling methods.

Exposure Fusion

Abstract: We propose a technique for fusing a bracketed exposure sequence into a high quality image, without converting to HDR first. Skipping the physically-based HDR assembly step simplifies the acquisition pipeline. This avoids camera response curve calibration and is computationally efficient. It also allows for including flash images in the sequence. Our technique blends multiple exposures, guided by simple quality measures like saturation and contrast. This is done in a multiresolution fashion to account for the brightness variation in the sequence. The resulting image quality is comparable to existing tone mapping operators.

Ldr2Hdr: on-the-fly reverse tone mapping of legacy video and photographs

Abstract: New generations of display devices promise to provide significantly improved dynamic range over conventional display technology. In the long run, evolving camera technology and file formats will provide high fidelity content for these display devices. In the near term, however, the vast majority of images and video will only be available in low dynamic range formats. In this paper we describe a method for boosting the dynamic range of legacy video and photographs for viewing on high dynamic range displays. Our emphasis is on real-time processing of video streams, such as web streams or the signal from a DVD player. We place particular emphasis on robustness of the method, and its ability to deal with a wide range of content without user adjusted parameters or visible artifacts. The method can be implemented on both graphics hardware and on signal processors that are directly integrated in the HDR displays.

Do HDR displays support LDR content?: a psychophysical evaluation

Abstract: The development of high dynamic range (HDR) imagery has brought us to the verge of arguably the largest change in image display technologies since the transition from black-and-white to color television. Novel capture and display hardware will soon enable consumers to enjoy the HDR experience in their own homes. The question remains, however, of what to do with existing images and movies, which are intrinsically low dynamic range (LDR). Can this enormous volume of legacy content also be displayed effectively on HDR displays? We have carried out a series of rigorous psychophysical investigations to determine how LDR images are best displayed on a state-of-the-art HDR monitor, and to identify which stages of the HDR imaging pipeline are perceptually most critical. Our main findings are: (1) As expected, HDR displays outperform LDR ones. (2) Surprisingly, HDR images that are tone-mapped for display on standard monitors are often no better than the best single LDR exposure from a bracketed sequence. (3) Most importantly of all, LDR data does not necessarily require sophisticated treatment to produce a compelling HDR experience. Simply boosting the range of an LDR image linearly to fit the HDR display can equal or even surpass the appearance of a true HDR image. Thus the potentially tricky process of inverse tone mapping can be largely circumvented.

Evaluating HDR rendering algorithms

Abstract: A series of three experiments has been performed to test both the preference and accuracy of high dynamic-range (HDR) rendering algorithms in digital photography application. The goal was to develop a methodology for testing a wide variety of previously published tone-mapping algorithms for overall preference and rendering accuracy. A number of algorithms were chosen and evaluated first in a paired-comparison experiment for overall image preference. A rating-scale experiment was then designed for further investigation of individual image attributes that make up overall image preference. This was designed to identify the correlations between image attributes and the overall preference results obtained from the first experiments. In a third experiment, three real-world scenes with a diversity of dynamic range and spatial configuration were designed and captured to evaluate seven HDR rendering algorithms for both of their preference and accuracy performance by comparing the appearance of the physical scenes and the corresponding tone-mapped images directly. In this series of experiments, a modified Durand and Dorsey's bilateral filter technique consistently performed well for both preference and accuracy, suggesting that it is a good candidate for a common algorithm that could be included in future HDR algorithm testing evaluations. The results of these experiments provide insight for understanding of perceptual HDR image rendering and should aid in design strategies for spatial processing and tone mapping. The results indicate ways to improve and design more robust rendering algorithms for general HDR scenes in the future. Moreover, the purpose of this research was not simply to find out the “best” algorithms, but rather to find a more general psychophysical experiment based methodology to evaluate HDR image-rendering algorithms. This paper provides an overview of the many issues involved in an experimental framework that can be used for these evaluations.

Model of retinal local adaptation for the tone mapping of color filter array images

Abstract: We present a tone mapping algorithm that is derived from a model of retinal processing. Our approach has two major improvements over existing methods. First, tone mapping is applied directly on the mosaic image captured by the sensor, analogous to the human visual system that applies a nonlinearity to the chromatic responses captured by the cone mosaic. This reduces the number of necessary operations by a factor 3. Second, we introduce a variation of the center/surround class of local tone mapping algorithms, which are known to increase the local contrast of images but tend to create artifacts. Our method gives a good improvement in contrast while avoiding halos and maintaining good global appearance. Like traditional center/surround algorithms, our method uses a weighted average of surrounding pixel values. Instead of being used directly, the weighted average serves as a variable in the Naka-Rushton equation, which models the photoreceptors' nonlinearity. Our algorithm provides pleasing results on various images with different scene content and dynamic range.

Digital bas-relief from 3D scenes

Abstract: We present a system for semi-automatic creation of bas-relief sculpture. As an artistic medium, relief spans the continuum between 2D drawing or painting and full 3D sculpture. Bas-relief (or low relief) presents the unique challenge of squeezing shapes into a nearly-flat surface while maintaining as much as possible the perception of the full 3D scene. Our solution to this problem adapts methods from the tone-mapping literature, which addresses the similar problem of squeezing a high dynamic range image into the (low) dynamic range available on typical display devices. However, the bas-relief medium imposes its own unique set of requirements, such as maintaining small, fixed-size depth discontinuities. Given a 3D model, camera, and a few parameters describing the relative attenuation of different frequencies in the shape, our system creates a relief that gives the illusion of the 3D shape from a given vantage point while conforming to a greatly compressed height.

Do HDR displays support LDR content

Abstract: The development of high dynamic range (HDR) imagery has brought us to the verge of arguably the largest change in image display technologies since the transition from black-and-white to color telev...

Tone mapping for high-dynamic range displays

Abstract: We address the problem of re-rendering images to high dynamic range (HDR) displays, which were originally tone-mapped to standard displays. As these new HDR displays have a much larger dynamic range than standard displays, an image rendered to standard monitors is likely to look too bright when displayed on a HDR monitor. Moreover, because of the operations performed during capture and rendering to standard displays, the specular highlights are likely to have been clipped or compressed, which causes a loss of realism. We propose a tone scale function to re-render images first tone-mapped to standard displays, that focuses on the representation of specular highlights. The shape of the tone scale function depends on the segmentation of the input image into its diffuse and specular components. In this article, we describe a method to perform this segmentation automatically. Our method detects specular highlights by using two low-pass filters of different sizes combined with morphological operators. The results show that our method successfully detects small and middle sized specular highlights. The locations of specular highlights define a mask used for the construction of the tone scale function. We then propose two ways of applying the tone scale, the global version that applies the same curve to each pixel in the image and the local version that uses spatial information given by the mask to apply the tone scale differently to diffuse and to specular pixels.

A framework for inverse tone mapping

Abstract: In recent years many tone mapping operators (TMOs) have been presented in order to display high dynamic range images (HDRI) on typical display devices. TMOs compress the luminance range while trying to maintain contrast. The inverse of tone mapping, inverse tone mapping, expands a low dynamic range image (LDRI) into an HDRI. HDRIs contain a broader range of physical values that can be perceived by the human visual system. We propose a new framework that approximates a solution to this problem. Our framework uses importance sampling of light sources to find the areas considered to be of high luminance and subsequently applies density estimation to generate an expand map in order to extend the range in the high luminance areas using an inverse tone mapping operator. The majority of today’s media is stored in the low dynamic range. Inverse tone mapping operators (iTMOs) could thus potentially revive all of this content for use in high dynamic range display and image based lighting (IBL). Moreover, we show another application that benefits quick capture of HDRIs for use in IBL.

High Dynamic Range Imaging Pipeline : Perception-Motivated Representation of Visual Content

Abstract: The advances in high dynamic range (HDR) imaging, especially in the display and camera technology, have a significant impact on the existing imaging systems. The assumptions of the traditional low-dynamic range imaging, designed for paper print as a major output medium, are ill suited for the range of visual material that is shown on modern displays. For example, the common assumption that the brightest color in an image is white can be hardly justified for high contrast LCD displays, not to mention next generation HDR displays, that can easily create bright highlights and the impression of self-luminous colors. We argue that high dynamic range representation can encode images regardless of the technology used to create and display them, with the accuracy that is only constrained by the limitations of the human eye and not a particular output medium. To facilitate the research on high dynamic range imaging, we have created a software package (http://pfstools.sourceforge.net/) capable of handling HDR data on all stages of image and video processing. The software package is available as open source under the General Public License and includes solutions for high quality image acquisition from multiple exposures, a range of tone mapping algorithms and a visual difference predictor for HDR images. Examples of shell scripts demonstrate how the software can be used for processing single images as well as video sequences.

High Dynamic Range Video

Abstract: As new displays and cameras offer enhanced color capabilities, there is a need to extend the precision of digital content. High Dynamic Range (HDR) imaging encodes images and video with higher than normal 8 bit-per-color-channel precision, enabling representation of the complete color gamut and the full visible range of luminance.However, to realize transition from the traditional toHDRimaging, it is necessary to develop imaging algorithms that work with the high-precision data. Tomake such algorithms effective and feasible in practice, it is necessary to take advantage of the limitations of the human visual system by aligning the data shortcomings to those of the human eye, thus limiting storage and processing precision. Therefore, human visual perception is the key component of the solutions we discuss in this book. This book presents a complete pipeline forHDR image and video processing fromacquisition, through compression and quality evaluation, to display. At the HDR image and video acquisition stage specialized HDR sensors or multi-exposure techniques suitable for traditional cameras are discussed. Then, we present a practical solution for pixel values calibration in terms of photometric or radiometric quantities, which are required in some technically oriented applications. Also, we cover the problem of efficient image and video compression and encoding either for storage or transmission purposes, including the aspect of backward compatibility with existing formats. Finally, we review existing HDR display technologies and the associated problems of image contrast and brightness adjustment. For this purpose tone mapping is employed to accommodate HDR content to LDR devices. Conversely, the so-called inverse tone mapping is required to upgrade LDR content for displaying on HDR devices. We overview HDR-enabled image and video quality metrics, which are needed to verify algorithms at all stages of the pipeline. Additionally, we cover successful examples of the HDR technology applications, in particular, in computer graphics and computer vision. The goal of this book is to present all discussed components of the HDR pipeline with the main focus on video. For some pipeline stages HDR video solutions are either not well established or do not exist at all, in which case we describe techniques for single HDR images. In such cases we attempt to select the techniques, which can be extended into temporal domain. Whenever needed, relevant background information on human perception is given, which enables better understanding of the design choices behind the discussed algorithms and HDR equipment. Table of Contents: Introduction / Representation of an HDR Image / HDR Image and Video Acquisition / HDR Image Quality / HDR Image, Video, and Texture Compression / Tone Reproduction / HDR Display Devices / LDR2HDR: Recovering Dynamic Range in Legacy Content / HDRI in Computer Graphics / Software

Messaging method and system for tone mapping, image reception device, image transmission device, and program

Abstract: PROBLEM TO BE SOLVED: To provide a method and system for signaling tone map data to an image receiver. SOLUTION: The system includes an image reception part 142 for receiving an image and for receiving a message, a map selection part 148 for selecting a first tone map for application to the image from a plurality of tone maps for use in converting an image from a first dynamic range to a second dynamic range, and a tone mapping part 144B for applying the first tone map to the image to produce a tone-mapped image. The message includes the plurality of tone maps, each of which is defined by a tone map model identifier and at least one tone map model parameter and is associated with a tone map identifier which identifies the purpose of the tone map. COPYRIGHT: (C)2008,JPO&INPIT

An FPGA-based architecture for a local tone-mapping operator

Abstract: This paper presents an FPGA-based architecture for local tone mapping of gray scale high dynamic range images. The architecture is described in VHDL and has been synthesized using Altera Quartus tools. It achieves an operating frequency consistent with a video rate of 60 frames per second for a frame of 1,024 × 768 pixels. The proposed architecture is a modification of the nine-scale Reinhard operator. Approximations to the original Reinhard operator ensure that the operator is amenable to implementation in hardware. A peak signal-to-noise ratio study shows that our fixed-point hardware approximation produces results similar to a floating-point original.

Gradient Domain Tone Mapping of High Dynamic Range Videos

Abstract: A gradient domain tone mapping algorithm is proposed to display high dynamic range (HDR) video sequences in low dynamic range (LDR) devices in this work. The proposed algorithm obtains a pixelwise motion vector field and incorporates the motion information into the Poisson equation. Then, by attenuating large spatial gradients, the proposed algorithm can yield a high-quality tone-mapped result without flickering artifacts. Simulation results show that the proposed algorithm provides a better performance than the frame-based method, which processes each frame independently.

Contrast Restoration by Adaptive Countershading

Abstract: We address the problem of communicating contrasts in images degraded with respect to their original due to processing with computer graphics algorithms. Such degradation can happen during the tone mapping of high dynamic range images, or while rendering scenes with low contrast shaders or with poor lighting. Inspired by a family of known perceptual illusions: Craik-O’Brien-Cornsweet, we enhance contrasts by modulating brightness at the edges to create countershading proles. We generalize unsharp masking by coupling it with a multi-resolution local contrast metric to automatically create the countershading proles from the sub-band components which are individually adjusted to each corrected feature to best enhance contrast with respect to the reference. Additionally, we employ a visual detection model to assure that our enhancements are not perceived as objectionable halo artifacts. The overall appearance of images remains mostly unchanged and the enhancement is achieved within the available dynamic range. We use our method to post-correct tone mapped images and improve images using their depth information.

Method of displaying a low dynamic range image in a high dynamic range

Abstract: A method of increasing the dynamic range of an image comprising a plurality of pixels each having a luminance value within a first luminance dynamic range. The method includes determining a background luminance value for each pixel of the image and determining a minimum and a maximum of the background luminance values. A conversion factor is then determined for each pixel of the image based on the minimum and maximum of the background luminance values. The image id converted from the first luminance dynamic range to a second luminance dynamic range by multiplying the luminance value of each pixel of the image by its conversion factor.

Image Display Algorithms for High and Low Dynamic Range Display Devices

Abstract: — With interest in high-dynamic-range imaging mounting, techniques for displaying such images on conventional display devices are gaining in importance. Conversely, high-dynamic-range display hardware is creating the need for display algorithms that prepare images for such displays. In this paper, the current state of the art in dynamic-range reduction and expansion is reviewed, and in particular the theoretical and practical need to structure tone reproduction as a combination of a forward and a reverse pass is passed.

Method for dynamic range editing

Abstract: A method of displaying a high dynamic range image, comprising receiving the high dynamic range image, calculating a first set of tone mapping parameters as a function of the high dynamic range image, sub-sampling the first set of tone mapping parameters at a first resolution to create a first sub-sampled parameter set, creating a first tone-mapped image by processing the high dynamic range image as a function of the first sub-sampled parameter set, and displaying the first tone-mapped image. A method of composting a plurality of versions of an image to create the high dynamic range image is also disclosed such that the compositing may be modified as a function of received user input.

Viewing wide angle images using dynamic tone mapping

Abstract: A dynamic tone mapping technique is presented that produces a local tone map for a sub-image of a wide-angle, high dynamic range (HDR), which is used in rendering the sub-image for display. The technique generally involves first computing a global tone map of the wide-angle, HDR image in advance of rendering the sub-image. The global tone map is then used during rendering to compute a local tone map based on the average luminance and contrast of the pixels of the sub-image. In addition, the sub-image can be tone mapped as part of the rendering of a sequence of sub-images during a viewer-executed panning and/or zooming session. In this case, the local tone maps can be kept from changing too rapidly by adding a hysteresis feature to smooth out the intensity changes between successive sub-images.

High-dynamic-range (HDR) vision : microelectronics, image processing, computer graphics

Abstract: The Eye and High-Dynamic-Range Vision- The High-Dynamic-Range Sensor- HDR Image Noise- High-Dynamic-Range Contrast and Color Management- HDR Video Cameras- Lenses for HDR Imaging- HDRC Cameras for High-Speed Machine Vision- HDR Vision for Driver Assistance- Miniature HDRC Cameras for Endoscopy- HDR Sub-retinal Implant for the Vision Impaired- HDR Tone Mapping- HDR Image and Video Compression- HDR Applications in Computer Graphics- High-Dynamic Range Displays

Design and Implementation of a Real-Time Global Tone Mapping Processor for High Dynamic Range Video

Abstract: As the development in high dynamic range (HDR) video capture technologies, the bit-depth video encoding and decoding has become an interesting topic. In this paper, we show that the real-time HDR video display is possible. A tone mapping based HDR video architecture pipelined with a video CODEC is presented. The HDR video is compressed by the tone mapping processor. The compressed HDR video can be encoded and decoded by the video standards, such as MPEG2, MPEG4 or H.264 for transmission and display. We propose and implement a modified photographic tone mapping algorithm for the tone mapping processor. The required luminance wordlength in the processor is analyzed and the quantization error is estimated. We also develop the digit-by-digit exponent and logarithm hardware architecture for the tone mapping processor. The synthesized results show that our real-time tone mapping processor can process a NTSC video with 720*480 resolution at 30 frames per second.

Block-Based Gradient Domain High Dynamic Range Compression Design for Real-Time Applications

Abstract: Due to progress in high dynamic range (HDR) capture technologies, the HDR image or video display on conventional LCD devices has become an important topic. Many tone mapping algorithms are proposed for rendering HDR images on conventional displays, but intensive computation time makes them impractical for video applications. In this paper, we present a real-time block-based gradient domain HDR compression for image or video applications. The gradient domain HDR compression is selected as our tone mapping scheme for its ability to compress and preserve details. We divide one HDR image/frame into several equal blocks and process each by the modified gradient domain HDR compression. The gradients of smaller magnitudes are attenuated less in each block to maintain local contrast and thus expose details. By solving the Poisson equation on the attenuated gradient field block by block, we are able to reconstruct a low dynamic range image. A real-time Discrete Sine Transform (DST) architecture is proposed and developed to solve the Poisson equation. Our synthesis results show that our DST Poisson solver can run at 50 MHz clock and consume area of 9 mm2 under TSMC 0.18 um technology.

Display methods for high dynamic range images and user interfaces for the same

Abstract: Display methods and interfaces for high dynamic range images. High dynamic range images and corresponding histograms are displayed on a display device. The histogram panel displays low dynamic range and high dynamic range data of the images simultaneously. The low dynamic range and high dynamic range data is brightness level of the image pixels.

A real-time FPGA-based architecture for a Reinhard-like tone mapping operator

Abstract: This paper presents a field-programmable gate array-based hardware architecture for a Reinhard-like tone mapping operator. Modifications to the original Reinhard operator were done to ensure that the operator is amenable to implementation in hardware. The architecture is described in VHDL and has been synthesized using Altera Quartus tools. It achieves an operating frequency consistent with a video rate of 60 frames per second for a frame of 1024x768 pixels. The quality of the implementation is measured using peak signal-to-noise ratios on testbench images.

High dynamic range image tone mapping and retexturing using fast trilateral filtering

Abstract: Using fast trilateral filtering we present a novel tone mapping and retexturing method for high dynamic range (HDR) images. Our new trilateral filtering-based tone mapping is about seven to ten times faster than that in [3]. Firstly, a novel tone mapping algorithm for HDR images is presented. It is based on fast bilateral filtering and two newly developed filters: the quasi-Cauchy function kernel filter and the fourth degree Taylor polynomial kernel filter. Secondly, a new gradient-based image retexturing method is introduced, which consists of three steps: 1) converting HDR images into low dynamic range (LDR) images using our fast trilateral filtering-based tone mapping method; 2) recovering the gradient luminance maps for the region to be retextured; 3) reconstructing the final retextured image by solving the Poisson equation. The proposed approach is suitable for HDR image tone mapping and retexturing, and experimental results have demonstrated the satisfactory performance of our method.