Camera sensors can only capture a limited range of luminance simultaneously, and in order to create high dynamic range (HDR) images a set of different exposures are typically combined. In this paper we address the problem of predicting information that have been lost in saturated image areas, in order to enable HDR reconstruction from a single exposure. We show that this problem is well-suited for deep learning algorithms, and propose a deep convolutional neural network (CNN) that is specifically designed taking into account the challenges in predicting HDR values. To train the CNN we gather a large dataset of HDR images, which we augment by simulating sensor saturation for a range of cameras. To further boost robustness, we pre-train the CNN on a simulated HDR dataset created from a subset of the MIT Places database. We demonstrate that our approach can reconstruct high-resolution visually convincing HDR results in a wide range of situations, and that it generalizes well to reconstruction of images captured with arbitrary and low-end cameras that use unknown camera response functions and post-processing. Furthermore, we compare to existing methods for HDR expansion, and show high quality results also for image based lighting. Finally, we evaluate the results in a subjective experiment performed on an HDR display. This shows that the reconstructed HDR images are visually convincing, with large improvements as compared to existing methods.

HDR image reconstruction from a single exposure using deep CNNs

digital video and hdtv algorithms and interfaces is available in our book collection an online access to it is set as public so you can get it instantly. Our book servers saves in multiple countries, allowing you to get the most less latency time to download any of our books like this one. Merely said, the digital video and hdtv algorithms and interfaces is universally compatible with any devices to read.

Digital Video And Hdtv Algorithms And Interfaces

There is disclosed a method, an apparatus, a server, a client and a non-transitory computer readable medium comprising a computer program stored therein for motion compensated video coding and decoding. Texture block motion information is used to derive disparity/depth motion information. Alternatively, disparity/depth motion information is used to derive texture block motion information.

An apparatus, a method and a computer program for video coding and decoding

Tone mapping aims to reproduce a standard dynamic range image from a high dynamic range image with visual information preserved. State-of-the-art tone mapping algorithms mostly decompose an image into a base layer and a detail layer, and process them accordingly. These methods may have problems of halo artifacts and over-enhancement, due to the lack of proper priors imposed on the two layers. In this paper, we propose a hybrid l1-l0 decomposition model to address these problems. Specifically, an l1 sparsity term is imposed on the base layer to model its piecewise smoothness property. An l0 sparsity term is imposed on the detail layer as a structural prior, which leads to piecewise constant effect. We further propose a multiscale tone mapping scheme based on our layer decomposition model. Experiments show that our tone mapping algorithm achieves visually compelling results with little halo artifacts, outperforming the state-of-the-art tone mapping algorithms in both subjective and objective evaluations.

/pdf/a-hybrid-l1-l0-layer-decomposition-model-for-tone-mapping-502f2omuqi.pdf

A Hybrid l1-l0 Layer Decomposition Model for Tone Mapping

One of the main goals of digital imagery is to improve the capture and the reproduction of real or synthetic scenes on display devices with restricted capabilities. Standard imagery techniques are limited with respect to the dynamic range that they can capture and reproduce. High Dynamic Range (HDR) imagery aims at overcoming these limitations by capturing, representing and displaying the physical value of light measured in a scene. However, current commercial displays will not vanish instantly hence backward compatibility between HDR content and those displays is required. This compatibility is ensured through an operation called tone mapping that retargets the dynamic range of HDR content to the restricted dynamic range of a display device. Although many tone mapping operators exist, they focus mostly on still images. The challenges of tone mapping HDR videos are more complex than those of still images since the temporal dimensions is added. In this work, the focus was on the preservation of temporal coherency when performing video tone mapping. Two main research avenues are investigated: the subjective quality of tone mapped video content and their compression efficiency. Indeed, tone mapping independently each frame of a video sequence leads to temporal artifacts. Those artifacts impair the visual quality of the tone mapped video sequence and need to be reduced. Through experimentations with HDR videos and Tone Mapping Operators (TMOs), we categorized temporal artifacts into six categories. We tested video tone mapping operators (techniques that take into account more than a single frame) on the different types of temporal artifact and we observed that they could handle only three out of the six types. Consequently, we designed a post-processing technique that adapts to any tone mapping operator and reduces the three types of artifact not dealt with. A subjective evaluation reported that our technique always preserves or increases the subjective quality of tone mapped content for the sequences and TMOs tested. The second topic investigated was the compression of tone mapped video content. So far, work on tone mapping and video compression focused on optimizing a tone map curve to achieve high compression ratio. These techniques changed the rendering of the video to reduce its entropy hence removing any artistic intent or constraint on the final results. That is why, we proposed a technique that reduces the entropy of a tone mapped video without altering its rendering.

/pdf/temporal-coherency-for-video-tone-mapping-qyorjws5fd.pdf

Temporal coherency for video tone mapping

High-dynamic-range (HDR) technology has attracted a lot of attention recently, especially in commercial trade shows such as the Consumer Electronics Show, the National Association of Broadcasters Show, the International Broadcasting Convention, and Internationale Funkausstellung Berlin. However, a great deal of mystery still surrounds this new evolution in digital media. In a nutshell, HDR technology aims at capturing, distributing, and displaying a range of luminance and color values that better correspond to what the human eye can perceive. Here, the term luminance stands for the photometric quantity of light arriving at the human eye measured in candela per square meter or nits. The color refers to all the weighted combinations of spectral wavelengths, expressed in nanometers (nm), emitted by the sun that are visible by the human eye (see Figure 1). The human eye can perceive a dynamic range of over 14 orders of magnitude (i.e., the difference in powers of ten between highest and lowest luminance value) in the real world through adaptation. However, at a single adaptation time, the human eye can only resolve up to five orders of magnitude, as illustrated in Figure 2. Dynamic range denotes the ratio between the highest and lowest luminance value. As reported in Table 1, there are different interpretations for dynamic range, depending on the application. For instance, in photography, dynamic range is measured in terms of f-stops, which correspond to the number of times that the light intensity can be doubled.

Demystifying High-Dynamic-Range Technology: A new evolution in digital media.

Tone Mapping Operators (TMOs) compress High Dynamic Range (HDR) contents to address Low Dynamic Range (LDR) displays. While many solutions have been designed over the last decade, only few of them can cope with video sequences. Indeed, these TMOs tone map each frame of a video sequence separately, which results in temporal incoherency. Two main types of temporal incoherency are usually considered: flickering artifacts and temporal brightness incoherency. While the reduction of flickering artifacts has been well studied, less work has been performed on brightness incoherency. In this paper, we propose a method that aims at preserving spatio-temporal brightness coherency when tone mapping video sequences. Our technique computes HDR video zones which are constant throughout a sequence, based on the luminance of each pixel. Our method aims at preserving the brightness coherency between the brightest zone of the video and each other zone. This technique adapts to any TMO and results show that it preserves well spatio-temporal brightness coherency. We validate our method using a subjective evaluation. In addition, unlike local TMOs, our method, when applied to still images, is capable of ensuring spatial brightness coherency. Finally, it also preserves video fade effects commonly used in post-production.

Zonal brightness coherency for video tone mapping

Traditional Low Dynamic Range (LDR) color spaces encode a small fraction of the visible color gamut, which does not encompass the range of colors produced on upcoming High Dynamic Range (HDR) displays. Future imaging systems will require encoding much wider color gamut and luminance range. Such wide color gamut can be represented using floating point HDR pixel values but those are inefficient to encode. They also lack perceptual uniformity of the luminance and color distribution, which is provided (in approximation) by most LDR color spaces. Therefore, there is a need to devise an efficient, perceptually uniform and integer valued representation for high dynamic range pixel values. In this paper we evaluate several methods for encoding colour HDR pixel values, in particular for use in image and video compression. Unlike other studies we test both luminance and color difference encoding in a rigorous 4AFC threshold experiments to determine the minimum bit-depth required. Results show that the Perceptual Quantizer (PQ) encoding provides the best perceptual uniformity in the considered luminance range, however the gain in bit-depth is rather modest. More significant difference can be observed between color difference encoding schemes, from which YDuDv encoding seems to be the most efficient.

https://www.cl.cam.ac.uk/~rkm38/pdfs/boitard14hdr_colorspace.pdf

Evaluation of color encodings for high dynamic range pixels

High Dynamic Range (HDR) imaging is capable of delivering a wider range of luminance and color gamut compared to Standard Dynamic Range (SDR), offering to viewers a visual quality of experience close to that of real-life. In this study, we evaluate the quality of coded original HDR streams and HDR streams reconstructed from SDR videos and metadata, both compressed by the HEVC standard. Our evaluations have shown that the single HDR approach is largely preferred over the SDR counterpart.

/pdf/compression-efficiency-of-hdr-ldr-content-498n2ovfb1.pdf

Ronan Boitard

Papers

Temporal coherency for video tone mapping

Demystifying High-Dynamic-Range Technology: A new evolution in digital media.

Zonal brightness coherency for video tone mapping

Evaluation of color encodings for high dynamic range pixels

Compression efficiency of HDR/LDR content