scispace - formally typeset
Search or ask a question

Showing papers on "Tone mapping published in 2020"


Journal ArticleDOI
TL;DR: DeepTMO as discussed by the authors proposes a conditional generative adversarial network (cGAN) to learn to adapt to vast scenic-content (e.g., outdoor, indoor, human, structures, etc.) and tackles the HDR related scene-specific challenges such as contrast and brightness, while preserving the fine-grained details.
Abstract: A computationally fast tone mapping operator (TMO) that can quickly adapt to a wide spectrum of high dynamic range (HDR) content is quintessential for visualization on varied low dynamic range (LDR) output devices such as movie screens or standard displays. Existing TMOs can successfully tone-map only a limited number of HDR content and require an extensive parameter tuning to yield the best subjective-quality tone-mapped output. In this paper, we address this problem by proposing a fast, parameter-free and scene-adaptable deep tone mapping operator (DeepTMO) that yields a high-resolution and high-subjective quality tone mapped output. Based on conditional generative adversarial network (cGAN), DeepTMO not only learns to adapt to vast scenic-content ( e.g. , outdoor, indoor, human, structures, etc.) but also tackles the HDR related scene-specific challenges such as contrast and brightness, while preserving the fine-grained details. We explore 4 possible combinations of Generator-Discriminator architectural designs to specifically address some prominent issues in HDR related deep-learning frameworks like blurring, tiling patterns and saturation artifacts. By exploring different influences of scales, loss-functions and normalization layers under a cGAN setting, we conclude with adopting a multi-scale model for our task. To further leverage on the large-scale availability of unlabeled HDR data, we train our network by generating targets using an objective HDR quality metric, namely Tone Mapping Image Quality Index (TMQI). We demonstrate results both quantitatively and qualitatively, and showcase that our DeepTMO generates high-resolution, high-quality output images over a large spectrum of real-world scenes. Finally, we evaluate the perceived quality of our results by conducting a pair-wise subjective study which confirms the versatility of our method.

93 citations


Journal ArticleDOI
TL;DR: This work develops a unified two-pathway model inspired by the biological vision, especially the early visual mechanisms, which contributes to image enhancement tasks including low dynamic range image enhancement and high dynamic range (HDR) image tone mapping.
Abstract: Image enhancement is an important pre-processing step for many computer vision applications especially regarding the scenes in poor visibility conditions. In this work, we develop a unified two-pathway model inspired by the biological vision, especially the early visual mechanisms, which contributes to image enhancement tasks including low dynamic range (LDR) image enhancement and high dynamic range (HDR) image tone mapping. Firstly, the input image is separated and sent into two visual pathways: structure-pathway and detail-pathway, corresponding to the M- and P-pathway in the early visual system, which code the low- and high-frequency visual information, respectively. In the structure-pathway, an extended biological normalization model is used to integrate the global and local luminance adaptation, which can handle the visual scenes with varying illuminations. On the other hand, the detail enhancement and local noise suppression are achieved in the detail-pathway based on local energy weighting. Finally, the outputs of structure-and detail-pathway are integrated to achieve the low-light image enhancement. In addition, the proposed model can also be used for tone mapping of HDR images with some fine-tuning steps. Extensive experiments on three datasets (two LDR image datasets and one HDR scene dataset) show that the proposed model can handle the visual enhancement tasks mentioned above efficiently and outperform the related state-of-the-art methods.

43 citations


Proceedings ArticleDOI
14 Jun 2020
TL;DR: In this paper, a multiscale bandpass convolutional neural network (MBCNN) was proposed to solve both texture and color restoration problems in an end-to-end manner.
Abstract: Image demoireing is a multi-faceted image restoration task involving both texture and color restoration. In this paper, we propose a novel multiscale bandpass convolutional neural network (MBCNN) to address this problem. As an end-to-end solution, MBCNN respectively solves the two sub-problems. For texture restoration, we propose a learnable bandpass filter (LBF) to learn the frequency prior for moire texture removal. For color restoration, we propose a two-step tone mapping strategy, which first applies a global tone mapping to correct for a global color shift, then performs local fine tuning of the color per pixel. Through an ablation study, we demonstrate the effectiveness of the different components of MBCNN. Experimental results on two public datasets show that our method outperforms state-of-the-art methods by a large margin (more than 2dB in terms of PSNR).

25 citations


Posted Content
TL;DR: Experimental results on two public datasets show that the novel multiscale bandpass convolutional neural network (MBCNN) outperforms state-of-the-art methods by a large margin.
Abstract: Image demoireing is a multi-faceted image restoration task involving both texture and color restoration. In this paper, we propose a novel multiscale bandpass convolutional neural network (MBCNN) to address this problem. As an end-to-end solution, MBCNN respectively solves the two sub-problems. For texture restoration, we propose a learnable bandpass filter (LBF) to learn the frequency prior for moire texture removal. For color restoration, we propose a two-step tone mapping strategy, which first applies a global tone mapping to correct for a global color shift, and then performs local fine tuning of the color per pixel. Through an ablation study, we demonstrate the effectiveness of the different components of MBCNN. Experimental results on two public datasets show that our method outperforms state-of-the-art methods by a large margin (more than 2dB in terms of PSNR).

17 citations


Journal ArticleDOI
TL;DR: A fast global and locally adaptive tone mapping algorithm and its field-programmable gate array (FPGA) implementation that can effectively suppress noise and achieve fast smoothed local histogram estimation with fewer bins while maintaining high accuracy is presented.
Abstract: We present a fast global and locally adaptive tone mapping algorithm and its field-programmable gate array (FPGA) implementation. The specially designed tone mapping function, which is based on local histogram equalization, controls global, and local characteristics individually. In contrast to other tonemap operators, our algorithm manages light/dark halos separately and by using local tonemap function alone, it can effectively suppress noise. We validated the effectiveness of our algorithms using subjective and objective assessment. Using an average of the bins, we achieve fast smoothed local histogram estimation with fewer bins while maintaining high accuracy. Our new implementation method requires minimal data access and reduced memory as it operates with a downscaled frame size of 240 × 135 pixels. Relative local area size is 248 × 248 @Full-HD resolution (1920 × 1080). For low-latency pixel output, the system performs the tone mapping using pixel information from the previous frame. When we implemented the system on FPGA (TB-7K-325TIMG and Xilinx Kintex-7), we achieved lightweight hardware as the total usage rate is about 25% of the available FPGA resource. Using an online 1080p video we demonstrate, a real-time video processing using our hardware tone mapping system.

17 citations


Journal ArticleDOI
TL;DR: It is demonstrated that the GIF with weighted aggregation performs well in the fields of computational photography and image processing, including single image detail enhancement, tone mapping of high-dynamic-range images, single image haze removal, etc.
Abstract: As a local filter, the guided image filtering (GIF) suffers from halo artifacts. To address this issue, a novel weighted aggregating strategy is proposed in this paper. By introducing the weighted aggregation to GIF, the proposed method called WAGIF can achieve results with sharp edges and avoid/reduce halo artifacts. More specifically, compared to the weighted guided image filtering and the gradient domain guided image filtering, the proposed method can achieve both fine and coarse smoothing results in the flat areas while preserving edges. In addition, the complexity of the proposed approach is O(N) for an image with N pixels. It is demonstrated that the GIF with weighted aggregation performs well in the fields of computational photography and image processing, including single image detail enhancement, tone mapping of high-dynamic-range images, single image haze removal, etc.

17 citations


Journal ArticleDOI
Dae-Eun Kim1, Munchurl Kim1
TL;DR: This paper proposes a novel learning-based low-complexity RTM scheme that not only expands the suppressed dynamic ranges (DR) of the SDR videos (or images), but also effectively restores lost detail in the S DR videos.
Abstract: Although high dynamic range (HDR) display has become popular recently, the legacy content such as standard dynamic range (SDR) video is still in service and needs to be properly converted on HDR display devices. Therefore, it is desirable for HDR TV sets to have the capability of automatically converting input SDR video into HDR video, which is called reverse tone mapping (RTM). In this paper, we propose a novel learning-based low-complexity RTM scheme that not only expands the suppressed dynamic ranges (DR) of the SDR videos (or images), but also effectively restores lost detail in the SDR videos. Most existing conventional RTM schemes have focused on how to expand the DR of global contrast, resulting in limitations in recovering lost detail of SDR videos. On the other hand, the recent convolutional neural network-based approaches show promising results, but they are too complex to be applied on the users’ devices in practice. In this paper, our learning-based RTM scheme is computationally simple but effective in recovering lost detail. To learn the SDR-to-HDR relation, training “SDR-HDR” images are first separated into their base layer components and detail layer components by applying a guided filter. The detail layer components of the “SDR-HDR” pairs are used to train the SDR-to-HDR mapping. The mapping matrices are computed based on kernel ridge regression. In the meantime, the global contrast of the base layers is expanded by a nonlinear function that suppresses darker regions and amplifies brighter regions to fit the full DR of a target HDR display. To verify the effectiveness of our learning-based RTM scheme, we performed subjective quality assessment for images and videos. The experimental results show that our RTM scheme outperforms the existing RTM scheme with the successful restoration of lost detail in SDR images.

16 citations


Journal ArticleDOI
TL;DR: The proposed Tone-mapping algorithm is compared with state-of-the-art algorithms, using some well-known metrics that quantify the quality of tone-mapped images, and is found to have the best performance.
Abstract: A new tone-mapping algorithm is presented for visualization of high dynamic range (HDR) images on low dynamic range (LDR) displays. In the first step, the real-world pixel intensities of the HDR image are transformed to a perceptual domain using the perceptual-quantizer (PQ). This is followed by construction of the histogram of the luminance channel. Tone-mapping curve is generated from the cumulative histogram. It is known that histogram-based tone-mapping approaches can lead to excessive stretching of contrast in highly populated bins, whereas the pixels in sparse bins can suffer from excessive compression of contrast. We handle these issues by restricting the pixel counts in the histogram to remain below a defined limit, determined by a uniform distribution model. The proposed method is compared with state-of-the-art algorithms, using some well-known metrics that quantify the quality of tone-mapped images, and is found to have the best performance.

15 citations


Proceedings ArticleDOI
06 Jul 2020
TL;DR: This work captures a novel light-field dataset featuring both a high spatial resolution and a high dynamic range (HDR) to enable the community to research and develop efficient reconstruction and tone-mapping algorithms for a hyper-realistic visual experience.
Abstract: Light-field (LF) imaging has various advantages over the traditional 2D photography, providing angular information of the real world scene by separately recording light rays in different directions. Despite the directional light information which enables new capabilities such as depth estimation, post-capture refocusing, and 3D modelling, currently available light-field datasets are very restricted in terms of spatial-resolution and dynamic range. In this work, we address this problem by capturing a novel light-field dataset featuring both a high spatial resolution and a high dynamic range (HDR). This dataset should enable the community to research and develop efficient reconstruction and tone-mapping algorithms for a hyper-realistic visual experience. The dataset consists of six static light-fields that are captured by a high-quality digital camera mounted on two precise linear axes using exposure bracketing at each view point. To demonstrate the usefulness of such a dataset, we also performed a thorough analysis on local and global tone-mapping of natural data in the context of novel view-rendering. The rendered results are compared and evaluated both visually and quantitatively. To our knowledge, the recorded dataset is the first attempt to jointly capture high-resolution and HDR light-fields.

11 citations


Proceedings ArticleDOI
01 Dec 2020
TL;DR: In this paper, a conditional generative adversarial network (GAN) is used to build an adversarial and adaptive tone mapping operator (adTMO) that converts HDR into low dynamic range (LDR) images.
Abstract: This work addresses tone mapping, a common approach to convert high dynamic range (HDR) images into low dynamic range (LDR) images. We approach this problem by using adaptive tone mapping. We propose to deploy a conditional generative adversarial networks to build an adversarial and adaptive tone mapping operator (adTMO) that converts HDR into LDR images. We use an objective quality metric called the Tone Mapped Image Quality Index (TMQI) to evaluate our adTMO. Trained with 256*256 images, adTMO is able to generate 256*256 and high-resolution 1024*2048 LDR images. Given 1024*2048 HDR images, TMQI of the generated LDR images reaches the value of 0.90, which outperforms all other contemporary tone mapping operators.

10 citations


Journal ArticleDOI
TL;DR: Experimental evaluation showed that the proposed automatic tone mapping operator (TMO) system supports HDR images while achieving satisfying results in terms of accuracy and computational cost.
Abstract: Various methods have been performed for the purpose of Low Dynamic Range (LDR) image retrieval. However, no major work concerning the High Dynamic Range (HDR) image indexing has been widely diffused yet. We therefore propose a method that tackles the problem of efficiently and accurately retrieving HDR images. The proposed system is based on a hybrid descriptor which combines two color features. The first one is histogram based on the hue–saturation–value (HSV) color space that approaches the perception of human vision, whereas the second comprises the first- and second-order moments of the color bands. As a dissimilarity measure, we retained the Manhattan distance. In the second part of our work, we proposed an automatic tone mapping operator (TMO) to get an overview on the result images by using Standard Dynamic Range (SDR) devices. Comparisons with recent state-of-the-art TMOs have shown that our TM method produces LDR images with adequate quality while maintaining low complexity. Finally, to test our retrieval system, we have created two databases. Experimental evaluation showed that our system supports HDR images while achieving satisfying results in terms of accuracy and computational cost.

Journal ArticleDOI
TL;DR: Experimental results in the public ESPL-LIVE HDR database show that the Pearson linear correlation coefficient and Spearman rank order correlation coefficient of the proposed method reach 0.8302 and 0.7887, respectively, which is superior to the state-of-the-art blind TMI quality assessment methods, and it means that the proposal is highly consistent with human visual perception.

Journal ArticleDOI
TL;DR: Extensive qualitative and quantitative evaluations on several HDR images encoded by two widely-used TFs confirm the strong HVS-imperceptibility capabilities of the method, as well as the robustness of the embedded watermarks to tone mapping, lossy compression, and common signal processing operations.
Abstract: This paper presents a watermarking method in the spatial domain with HVS-imperceptibility for High Dynamic Range (HDR) images. The proposed method combines the content readability afforded by invisible watermarking with the visual ownership identification afforded by visible watermarking. The HVS-imperceptibility is guaranteed thanks to a Luma Variation Tolerance (LVT) curve, which is associated with the transfer function (TF) used for HDR encoding and provides the information needed to embed an imperceptible watermark in the spatial domain. The LVT curve is based on the inaccuracies between the non-linear digital representation of the linear luminance acquired by an HDR sensor and the brightness perceived by the Human Visual System (HVS) from the linear luminance displayed on an HDR screen. The embedded watermarks remain imperceptible to the HVS as long as the TF is not altered or the normal calibration and colorimetry conditions of the HDR screen remain unchanged. Extensive qualitative and quantitative evaluations on several HDR images encoded by two widely-used TFs confirm the strong HVS-imperceptibility capabilities of the method, as well as the robustness of the embedded watermarks to tone mapping, lossy compression, and common signal processing operations.

Journal ArticleDOI
TL;DR: The proposed tone mapping method performs robustly well on a wide variety of images, providing competitive results against the state-of-the-art methods in terms of visual inspection, objective metrics and observer scores.
Abstract: The limited dynamic range of regular screens restricts the display of high dynamic range (HDR) images. Inspired by retinal processing mechanisms, we propose a tone mapping method to address this problem. In the retina, horizontal cells (HCs) adaptively adjust their receptive field (RF) size based on the local stimuli to regulate the visual signals absorbed by photoreceptors. Using this adaptive mechanism, the proposed method compresses the dynamic range locally in different regions, and has the capability of avoiding halo artifacts around the edges of high luminance contrast. Moreover, the proposed method introduces the center-surround antagonistic RF structure of bipolar cells (BCs) to enhance the local contrast and details. Extensive experiments show that the proposed method performs robustly well on a wide variety of images, providing competitive results against the state-of-the-art methods in terms of visual inspection, objective metrics and observer scores.

Journal ArticleDOI
TL;DR: A novel FPGA architecture of high dynamic range (HDR) video processing pipeline based on the capturing of a sequence of differently exposed images achieves real-time performance on full HD HDR video which overcomes state-of-the-art solutions that use local tone mapping and deghosting algorithm.
Abstract: This paper presents a novel FPGA architecture of high dynamic range (HDR) video processing pipeline, based on the capturing of a sequence of differently exposed images. An acquisition process enabling multi-exposure HDR as well as fast implementation of local tone mapping operator involving bilateral filtering is proposed. The HDR acquisition process is enhanced by the application of novel deghosting method, which is dedicated for hardware implementation and proposed in this paper. The hardware processing pipeline is designed with regards to efficiency and performance and the calculations are performed in fixed point arithmetic. The pipeline is suitable for programmable hardware (FPGA—Field Programmable Gate Arrays) implementation and it achieves real-time performance on full HD HDR video which overcomes state-of-the-art solutions that use local tone mapping and deghosting algorithm.

Journal ArticleDOI
Mingxing Jiang1, Liquan Shen1, Linru Zheng1, Min Zhao1, Xuhao Jiang1 
TL;DR: Experimental results show that the proposed tone-mapped images (TMIs) luminance partition model and corresponding quality measure outperforms the state-of-the-art no-reference (NR) methods.
Abstract: Tone mapping operators (TMOs) reproduce the high dynamic range (HDR) images on low dynamic range (LDR) consumer electronics devices such as monitors or printers. To accurately measure and compare the performance of different TMOs, this article proposes a tone-mapped images (TMIs) luminance partition model and corresponding quality measure. First, each tone-mapped (TM) image is segmented into highlight region (HR), dark region (DR) and midtone region (MR) based on luminance partition. Second, local entropies and contrast features are extracted in the HR and DR, and color-based features are captured in the MR. Meanwhile, the gray-level co-occurrence matrix (GLCM) and Canny operator are utilized to measure the microstructural distortions and halo effects, respectively. Finally, all extracted features are combined and trained together with subjective ratings to form a regression model using support vector regression (SVR). Experimental results show that the proposed method outperforms the state-of-the-art no-reference (NR) methods. Specifically, the spearman correlation coefficients (SRCC) values of our method reach 0.83 and 0.76 on the tone-mapped image database (TMID) and the ESPL-LIVE HDR database, respectively.

Journal ArticleDOI
TL;DR: A deep learning-based TM method for X-ray inspection that aims to compress the dynamic range while preserving the restored details and preventing halo artifacts and a dataset synthesis technique using the Beer-Lambert law for supervised learning of DR-Net.
Abstract: X-ray imaging is one of the most widely used security measures for maintaining airport and transportation security. Conventional X-ray imaging systems typically apply tone-mapping (TM) algorithms to visualize high-dynamic-range (HDR) X-ray images on a standard 8-bit display device. However, X-ray images obtained through traditional TM algorithms often suffer from halo artifacts or detail loss in inter-object overlapping regions, which makes it difficult for an inspector to detect unsafe or hazardous objects. To alleviate these problems, this article proposes a deep learning-based TM method for X-ray inspection. The proposed method consists of two networks called detail-recovery network (DR-Net) and TM network (TM-Net). The goal of DR-Net is to restore the details in the input HDR image, whereas TM-Net aims to compress the dynamic range while preserving the restored details and preventing halo artifacts. Since there are no standard ground-truth images available for the TM of X-ray images, we propose a novel loss function for unsupervised learning of TM-Net. We also introduce a dataset synthesis technique using the Beer-Lambert law for supervised learning of DR-Net. Extensive experiments comparing the performance of our proposed method with state-of-the-art TM methods demonstrate that the proposed method not only achieves visually compelling results but also improves the quantitative performance measures such as FSITM and HDR-VDP-2.2.

Journal ArticleDOI
01 Mar 2020-Optik
TL;DR: A defect detection method based on HDRI is proposed to solve the visual inspection problem of industrial parts with high-reflective surface by using adaptive tone mapping based on color correction model in gradient domain and adaptive threshold segmentation and Haar-like feature extraction algorithm.

Journal ArticleDOI
TL;DR: Results indicate that the proposed mantissa exponent-based algorithm provides visually pleasing results and preserves details of the original WDR image better than other similar algorithms and the hardware resources’ efficiency of the algorithm makes the system on chip implementation possible.
Abstract: The dynamic range of a scene is defined as the ratio between the maximum and minimum luminance in it. Wide dynamic range (WDR) means this ratio is so large that it exceeds the dynamic range of a traditional image sensor. Nowadays, WDR image sensors enable the capture of WDR scenes. However, the captured WDR image requires an additional tone mapping step to compress the high bit pixel of WDR image to low rate pixel so that it can be displayed on the screen. The tone mapping algorithm is mostly done in an image signal processor or with a specific software application. This brief proposes a tone mapping technique that is suitable for direct processing of the output of a WDR image sensor bitstream. The algorithm acquires statistics on the mantissa and exponent parts of the pixel value and then generates a refined histogram for tone mapping. Experiments that evaluate the image quality and hardware efficiency are carried out. The results indicate that the proposed mantissa exponent-based algorithm provides visually pleasing results and preserves details of the original WDR image better than other similar algorithms. The hardware resources’ efficiency of the algorithm makes the system on chip implementation possible.

Journal ArticleDOI
TL;DR: A completely blind image quality evaluator for tone-mapped images based on a multi-attribute feature extraction scheme that shows superior performance compared to the competing metrics is proposed.
Abstract: High dynamic range (HDR) imaging enables capturing a wide range of luminance levels existing in real-world scenes. While HDR capturing devices become widespread in the market, the display technology is yet limited in representing full luminance ranges and standard low dynamic range (LDR) displays are currently more prevalent. To visualize the HDR content on traditional displays, tone mapping (TM) operators are introduced that convert HDR content into LDR. The dynamic range compression and different processing steps during TM can lead to loss of scene details, as well as luminance and chrominance changes. Such signal deviations will affect image naturalness and consequently disturb the visual quality of experience. Therefore, research into objective methods for quality evaluation of tone-mapped images has received attention in recent years. In this paper, we proposed a completely blind image quality evaluator for tone-mapped images based on a multi-attribute feature extraction scheme. Due to the diversity of TM distortions, various image characteristics are taken into account to develop an effective metric. The features are designed by considering spectral and spatial entropy, detection probability of visual information, image exposure, sharpness, and color properties. The quality-relevant features are then fed into a machine-learning regression framework to pool a quality score. The validation tests on two benchmark datasets reveal the superior performance of the proposed approach compared to the competing metrics.

Journal ArticleDOI
16 Jul 2020-Sensors
TL;DR: This work proposes a method to generate a high dynamic range image from a single LDR image and introduces a technique for the matching between the histogram of a highynamic range (HDR) image and the original image.
Abstract: Extending the dynamic range can present much richer contrasts and physical information from the traditional low dynamic range (LDR) images. To tackle this, we propose a method to generate a high dynamic range image from a single LDR image. In addition, a technique for the matching between the histogram of a high dynamic range (HDR) image and the original image is introduced. To evaluate the results, we utilize the dynamic range for independent image quality assessment. It recognizes the difference in subtle brightness, which is a significant role in the assessment of novel lighting, rendering, and imaging algorithms. The results show that the picture quality is improved, and the contrast is adjusted. The performance comparison with other methods is carried out using the predicted visibility (HDR-VDP-2). Compared to the results obtained from other techniques, our extended HDR images can present a wider dynamic range with a large difference between light and dark areas.

Posted Content
TL;DR: This paper proposes a learning-based multimodal tone-mapping method, which not only achieves excellent visual quality but also explores the style diversity and shows that the proposed method performs favorably against state-of-the-art tone-Mapping algorithms both quantitatively and qualitatively.
Abstract: Tone-mapping plays an essential role in high dynamic range (HDR) imaging. It aims to preserve visual information of HDR images in a medium with a limited dynamic range. Although many works have been proposed to provide tone-mapped results from HDR images, most of them can only perform tone-mapping in a single pre-designed way. However, the subjectivity of tone-mapping quality varies from person to person, and the preference of tone-mapping style also differs from application to application. In this paper, a learning-based multimodal tone-mapping method is proposed, which not only achieves excellent visual quality but also explores the style diversity. Based on the framework of BicycleGAN, the proposed method can provide a variety of expert-level tone-mapped results by manipulating different latent codes. Finally, we show that the proposed method performs favorably against state-of-the-art tone-mapping algorithms both quantitatively and qualitatively.

Journal ArticleDOI
TL;DR: The recognition rate of computer vision algorithms is highly dependent on the image quality, and to enhance the visual quality of the images captured under high-dynamic range (HDR) scenes, a new approach is proposed.
Abstract: The recognition rate of computer vision algorithms is highly dependent on the image quality. To enhance the visual quality of the images captured under high-dynamic range (HDR) scenes, we propose a...

Journal ArticleDOI
23 Feb 2020
TL;DR: Experimental results demonstrate that the proposed fully-automatic inverse tone mapping operator based on mid-level mapping capable of real-time video processing outperforms the current state-of-the-art of simple inversetone mapping methods and its performance is similar to other more complex and time-consuming advanced techniques.
Abstract: High Dynamic Range (HDR) displays can show images with higher color contrast levels and peak luminosities than the common Low Dynamic Range (LDR) displays. However, most existing video content is recorded and/or graded in LDR format. To show LDR content on HDR displays, it needs to be up-scaled using a so-called inverse tone mapping algorithm. Several techniques for inverse tone mapping have been proposed in the last years, going from simple approaches based on global and local operators to more advanced algorithms such as neural networks. Some of the drawbacks of existing techniques for inverse tone mapping are the need for human intervention, the high computation time for more advanced algorithms, limited low peak brightness, and the lack of the preservation of the artistic intentions. In this paper, we propose a fully-automatic inverse tone mapping operator based on mid-level mapping capable of real-time video processing. Our proposed algorithm allows expanding LDR images into HDR images with peak brightness over 1000 nits, preserving the artistic intentions inherent to the HDR domain. We assessed our results using the full-reference objective quality metrics HDR-VDP-2.2 and DRIM, and carrying out a subjective pair-wise comparison experiment. We compared our results with those obtained with the most recent methods found in the literature. Experimental results demonstrate that our proposed method outperforms the current state-of-the-art of simple inverse tone mapping methods and its performance is similar to other more complex and time-consuming advanced techniques.

Journal ArticleDOI
TL;DR: In this paper, a hybrid deep emperor penguin classifier was proposed to accurately classify the tone mapped images for different visualisation applications, where a selective deep neural network was trained to predict the quality of a tone-mapped image.
Abstract: One of the main open challenges in visualisation applications such as cathode ray tube (CRT) monitor, liquid-crystal display (LCD), and organic light-emitting diode (OLED) display is the robustness for high dynamic range (HDR) environs. This is due to the imperfections in the sensor and the incapability to track interest points successfully because of the brightness constancy in visualisation applications. To address this problem, different tone mapping operators are required for visualising HDR images on standard displays. However, these standard displays have different dynamic ranges. Thus, there is a need for a new model to find the best quality tone mapped image for specific kinds of visualisation applications. The authors propose a hybrid deep emperor penguin classifier to accurately classify the tone mapped images for different visualisation applications. Here, a selective deep neural network is trained to predict the quality of a tone-mapped image. Based on this quality, a decision is made as to the suitability of the image for CRT monitor, LCD display or OLED display. Also, they evaluate the proposed model on the TMIQD database and the simulation results prove that the proposed model outperforms the state-of-the-art image quality assessment methods.

Journal ArticleDOI
TL;DR: A complete CMOS camera based on the HDR method is implemented, which produces a real-time HDR live video streams, and an ALTERA FPGA is the core processing unit of the entire camera, and it completes all the functional modules of the camera efficiently.
Abstract: To overcome the ghosting phenomenon of multi-exposure technology, a new high-dynamic range (HDR) image processing method is proposed in this paper, which combines the features from dual-gain channel images. Further, a complete CMOS camera based on the HDR method is implemented, which produces a real-time HDR live video streams. This camera can capture the details of bright and dark areas in the scene completely with an extended dynamic range up to 95 dB. An ALTERA FPGA is the core processing unit of the entire camera, and it completes all the functional modules of the camera efficiently, including dual-channel video capture, image caching, HDR synthesis and tone mapping. Finally, the real-time HDR video flow has a display resolution of 1920 × 1080 and a frame rate of 60 fps.

Journal ArticleDOI
TL;DR: A binary classification model is developed based on the determined features to classify if an image looks natural or unnatural, especially on tone-mapped images.

Patent
Oh Seung Bo1
30 Jun 2020
TL;DR: In this article, a tone-mapped image data is used to display the image based on the tone mappings of the image data to the tone mapping curve of a tone mapper.
Abstract: A display apparatus displaying, a method for controlling the display apparatus, and an image providing apparatus. The display apparatus includes a communicator configured to receive image data of an image and brightness information of the image, a processor configured to generate a tone mapping curve by using the brightness information and to apply the tone mapping curve to the image data to provide tone mapped image data, and a display configured to display the image based on the tone-mapped image data.

Journal ArticleDOI
TL;DR: Simulated glares are effective for highlighting small areas (of a few pixels) that may not be visible with conventional visualizations; through a controlled perception study, it is confirmed that glare is preattentive.
Abstract: We propose a photographic method to show scalar values of high dynamic range (HDR) by color mapping for 2D visualization. We combine (1) tone-mapping operators that transform the data to the display range of the monitor while preserving perceptually important features, based on a systematic evaluation, and (2) simulated glares that highlight high-value regions. Simulated glares are effective for highlighting small areas (of a few pixels) that may not be visible with conventional visualizations; through a controlled perception study, we confirm that glare is preattentive. The usefulness of our overall photographic HDR visualization is validated through the feedback of expert users.

Journal ArticleDOI
TL;DR: The proposed method combines lightness correction and amplification of high spatial frequency components to obtain images with optimized visibility for elderly persons based on the inverse characteristic function of the contrast sensitivity curve.
Abstract: In general, as people age the functions of their sensory organs deteriorate. For example, in the eye of an elderly person, the visual field becomes dark owing to a decrease in the amount of light reaching the retina. Moreover, objects appear blurred due to a decrease in contrast sensitivity. In particular, the contrast sensitivity decreases remarkably as the spatial frequency increases. In this study, we propose an image enhancement method that considers the visual characteristics of elderly persons. The proposed method combines lightness correction and amplification of high spatial frequency components to obtain images with optimized visibility for elderly persons. In particular, lightness enhancement that guarantees color gamut via tone mapping is conducted. In addition, image sharpening is applied based on the inverse characteristic function of the contrast sensitivity curve. The experiments were performed using several digital color images to confirm the effectiveness of the proposed method.