Naturalness Preserved Nonuniform Illumination Estimation for Image Enhancement Based on Retinex
TL;DR: A naturalness preserved illumination estimation algorithm based on the proposed joint edge-preserving filter which exploits all the constraints into the consideration and can achieve the adaptive smoothness of illumination beyond edges and ensure the range of the estimated illumination.
Abstract: Illumination estimation is important for image enhancement based on Retinex. However since illumination estimation is an ill-posed problem it is difficult to achieve accurate illumination estimation for nonuniform illumination images. The conventional illumination estimation algorithms fail to comprehensively take all the constraints into the consideration such as spatial smoothness sharp edges on illumination boundaries and limited range of illumination. Thus these algorithms cannot effectively and efficiently estimate illumination while preserving naturalness. In this paper we present a naturalness preserved illumination estimation algorithm based on the proposed joint edge-preserving filter which exploits all the abovementioned constraints. Moreover a fast estimation is implemented based on the box filter. Experimental results demonstrate that the proposed algorithm can achieve the adaptive smoothness of illumination beyond edges and ensure the range of the estimated illumination. When compared with other state-of-the-art algorithms it can achieve better quality from both subjective and objective aspects.
Citations
More filters
TL;DR: Experimental results on several public datasets demonstrate that the proposed Retinex-based low-light image enhancement method produces images with both higher visibility and better visual quality, which outperforms the state-of-the-art low- light enhancement methods in terms of several objective and subjective evaluation metrics.
Abstract: Low-light image enhancement is important for high-quality image display and other visual applications. However, it is a challenging task as the enhancement is expected to improve the visibility of an image while keeping its visual naturalness. Retinex-based methods have well been recognized as a representative technique for this task, but they still have the following limitations. First, due to less-effective image decomposition or strong imaging noise, various artifacts can still be brought into enhanced results. Second, although the priori information can be explored to partially solve the first issue, it requires to carefully model the priori by a regularization term and usually makes the optimization process complicated. In this paper, we address these issues by proposing a novel Retinex-based low-light image enhancement method, in which the Retinex image decomposition is achieved in an efficient semi-decoupled way. Specifically, the illumination layer $I$ is gradually estimated only with the input image $S$ based on the proposed Gaussian Total Variation model, while the reflectance layer $R$ is jointly estimated by $S$ and the intermediate $I$ . In addition, the imaging noise can be simultaneously suppressed during the estimation of $R$ . Experimental results on several public datasets demonstrate that our method produces images with both higher visibility and better visual quality, which outperforms the state-of-the-art low-light enhancement methods in terms of several objective and subjective evaluation metrics.
141 citations
Cites methods from "Naturalness Preserved Nonuniform Il..."
...In [11], an illumination estimation algorithm based on a joint edge-preserving filter is proposed for the naturalness-preserving Retinex decomposition....
[...]
TL;DR: A light enhancement net (LE-net) based on the convolutional neural network is developed based on a generation pipeline to transform daytime images to low-light images, and results showed that the LE-net was superior to the compared models, both qualitatively and quantitatively.
Abstract: Images of road scenes in low-light situations are lack of details which could increase crash risk of connected autonomous vehicles (CAVs). Therefore, an effective and efficient image enhancement model for low-light images is necessary for safe CAV driving. Though some efforts have been made, image enhancement still cannot be well addressed especially in extremely low light situations (e.g., in rural areas at night without street light). To address this problem, we developed a light enhancement net (LE-net) based on the convolutional neural network. Firstly, we proposed a generation pipeline to transform daytime images to low-light images, and then used them to construct image pairs for model development. Our proposed LE-net was then trained and validated on the generated low-light images. Finally, we examined the effectiveness of our LE-net in real night situations at various low-light levels. Results showed that our LE-net was superior to the compared models, both qualitatively and quantitatively.
108 citations
15 Oct 2018
TL;DR: This work addresses the problem of correcting the exposure of underexposed photos by casting the exposure correction problem as an illumination estimation optimization, where PBS is defined as three constraints for estimating illumination that can generate the desired result with even exposure, vivid color and clear textures.
Abstract: We address the problem of correcting the exposure of underexposed photos. Previous methods have tackled this problem from many different perspectives and achieved remarkable progress. However, they usually fail to produce natural-looking results due to the existence of visual artifacts such as color distortion, loss of detail, exposure inconsistency, etc. We find that the main reason why existing methods induce these artifacts is because they break a perceptually similarity between the input and output. Based on this observation, an effective criterion, termed as perceptually bidirectional similarity (PBS) is proposed. Based on this criterion and the Retinex theory, we cast the exposure correction problem as an illumination estimation optimization, where PBS is defined as three constraints for estimating illumination that can generate the desired result with even exposure, vivid color and clear textures. Qualitative and quantitative comparisons, and the user study demonstrate the superiority of our method over the state-of-the-art methods.
88 citations
TL;DR: A novel generation-and-fusion strategy is introduced, where the enhancements for slightly and heavily distorted images are carried out respectively in the two enhancing branches, followed by a self-adaptive attention unit to perform the final fusion.
Abstract: Images obtained under low-light conditions are usually accompanied by varied and highly unpredictable degradation. The uncertainty of the imaging environment makes the enhancement even more challenging. In this paper, we present a two-branch exposure-fusion network to tackle the problem of blind low-light image enhancement. In the first part of the paper, we provide a basic insight into the degradation mechanism of low-light images, and propose a quick and effective enhancement strategy by estimating the transfer function for varied illumination levels. To further deal with the challenge brought about by the blindness of input images, a novel generation-and-fusion strategy is then introduced, where the enhancements for slightly and heavily distorted images are carried out respectively in the two enhancing branches, followed by a self-adaptive attention unit to perform the final fusion. Moreover, a two-stage denoising strategy is also proposed to ensure effective noise reduction in a data-driven manner. To evaluate the performance of the proposed method, three commonly used datasets are adopted for quantitative evaluation and six for visual evaluation, where our method outperforms many of the existing state-of-the-art ones, showing great effectiveness and potential.
59 citations
TL;DR: The proposed method based on a multi-scale retinex with color restoration (MSRCR) of multi-channel convolution (MC) can outperform state-of-the-art methods in both qualitative and quantitative comparisons.
Abstract: In order to solve the problem of image degradation in foggy weather, a single image defogging method based on a multi-scale retinex with color restoration (MSRCR) of multi-channel convolution (MC) is proposed. The whole defogging process mainly consists of four key parts: estimation of illumination components, guided filter operation, reconstruction of fog-free images, and white balance operation. First, the multi-scale Gaussian kernels are employed to extract precise features to estimate the illumination component. After that, the MSRCR method is applied to enhance the global contrast, detail information, and color restoration of the image. Second, the smoothing constraints of both illumination component and reflected component are considered together by using the guided filter twice, thus the enhanced image satisfies the smoothing constraint and the noise in the enhanced image is reduced. Third, the enhanced image by the MSRCR and the image processed by the secondary guided filter are fused by linear weighting to reconstruct the final fog-free image. Finally, in order to eliminate the influence of illumination on the color of the defogged image, the final defogged image is processed by white balance. The experimental results demonstrated that the proposed method can outperform state-of-the-art methods in both qualitative and quantitative comparisons.
57 citations
Cites background from "Naturalness Preserved Nonuniform Il..."
...In the outdoor environment, visibility and contrast of a photograph will seriously reduce due to bad weather such as light, fog and haze [1]–[3]....
[...]
...Therefore, image defogging has become an important research direction, which has attracted more and more attention of researchers [1]–[6]....
[...]
References
More filters
TL;DR: The mathematics of a lightness scheme that generates lightness numbers, the biologic correlate of reflectance, independent of the flux from objects is described.
Abstract: Sensations of color show a strong correlation with reflectance, even though the amount of visible light reaching the eye depends on the product of reflectance and illumination. The visual system must achieve this remarkable result by a scheme that does not measure flux. Such a scheme is described as the basis of retinex theory. This theory assumes that there are three independent cone systems, each starting with a set of receptors peaking, respectively, in the long-, middle-, and short-wavelength regions of the visible spectrum. Each system forms a separate image of the world in terms of lightness that shows a strong correlation with reflectance within its particular band of wavelengths. These images are not mixed, but rather are compared to generate color sensations. The problem then becomes how the lightness of areas in these separate images can be independent of flux. This article describes the mathematics of a lightness scheme that generates lightness numbers, the biologic correlate of reflectance, independent of the flux from objects
3,480 citations
"Naturalness Preserved Nonuniform Il..." refers background in this paper
...In literatures [24], based on the assumption that bright areas of images are white surfaces or light sources as well as highlights, the so-called Max-RGB algorithm estimates the illumination from the maximum response of three color channels....
[...]
TL;DR: This paper extends a previously designed single-scale center/surround retinex to a multiscale version that achieves simultaneous dynamic range compression/color consistency/lightness rendition and defines a method of color restoration that corrects for this deficiency at the cost of a modest dilution in color consistency.
Abstract: Direct observation and recorded color images of the same scenes are often strikingly different because human visual perception computes the conscious representation with vivid color and detail in shadows, and with resistance to spectral shifts in the scene illuminant. A computation for color images that approaches fidelity to scene observation must combine dynamic range compression, color consistency-a computational analog for human vision color constancy-and color and lightness tonal rendition. In this paper, we extend a previously designed single-scale center/surround retinex to a multiscale version that achieves simultaneous dynamic range compression/color consistency/lightness rendition. This extension fails to produce good color rendition for a class of images that contain violations of the gray-world assumption implicit to the theoretical foundation of the retinex. Therefore, we define a method of color restoration that corrects for this deficiency at the cost of a modest dilution in color consistency. Extensive testing of the multiscale retinex with color restoration on several test scenes and over a hundred images did not reveal any pathological behaviour.
2,395 citations
"Naturalness Preserved Nonuniform Il..." refers background in this paper
...In order to improve the visual quality of images captured under non-uniform light condition, removing illumination [5]–[7] and remapping illumination [8]–[10] are common strategies....
[...]
...[5], [7] proposed to estimate illumination by the Gaussian filter....
[...]
TL;DR: A practical implementation of the retinex is defined without particular concern for its validity as a model for human lightness and color perception, and the trade-off between rendition and dynamic range compression that is governed by the surround space constant is described.
Abstract: The last version of Land's (1986) retinex model for human vision's lightness and color constancy has been implemented and tested in image processing experiments. Previous research has established the mathematical foundations of Land's retinex but has not subjected his lightness theory to extensive image processing experiments. We have sought to define a practical implementation of the retinex without particular concern for its validity as a model for human lightness and color perception. We describe the trade-off between rendition and dynamic range compression that is governed by the surround space constant. Further, unlike previous results, we find that the placement of the logarithmic function is important and produces best results when placed after the surround formation. Also unlike previous results, we find the best rendition for a "canonical" gain/offset applied after the retinex operation. Various functional forms for the retinex surround are evaluated, and a Gaussian form is found to perform better than the inverse square suggested by Land. Images that violate the gray world assumptions (implicit to this retinex) are investigated to provide insight into cases where this retinex fails to produce a good rendition.
1,674 citations
"Naturalness Preserved Nonuniform Il..." refers background or methods in this paper
...In order to improve the visual quality of images captured under non-uniform light condition, removing illumination [5]–[7] and remapping illumination [8]–[10] are common strategies....
[...]
...2 shows a group of illumination estimation processed by Gaussian filter [5] (GF), Wang algorithm [8], Shin algorithm [23] and brightpass bilateral filter [18] (BPBF)....
[...]
...[5], [7] proposed to estimate illumination by the Gaussian filter....
[...]
...According to the Retinex theory, the observed image is the product of reflectance and illumination as follow [5]:...
[...]
PARC1
TL;DR: Texture-map computations can be made tractable through use of precalculated tables which allow computational costs independent of the texture density, and the cost and performance of the new technique is compared to previous techniques.
Abstract: Texture-map computations can be made tractable through use of precalculated tables which allow computational costs independent of the texture density. The first example of this technique, the “mip” map, uses a set of tables containing successively lower-resolution representations filtered down from the discrete texture function. An alternative method using a single table of values representing the integral over the texture function rather than the function itself may yield superior results at similar cost. The necessary algorithms to support the new technique are explained. Finally, the cost and performance of the new technique is compared to previous techniques.
1,455 citations
"Naturalness Preserved Nonuniform Il..." refers methods in this paper
...Inspired by box filter [25], we can see that the most part of the calculation between the two adjacent points is repeated in the process of calculation....
[...]
TL;DR: Experiments on a number of challenging low-light images are present to reveal the efficacy of the proposed LIME and show its superiority over several state-of-the-arts in terms of enhancement quality and efficiency.
Abstract: When one captures images in low-light conditions, the images often suffer from low visibility. Besides degrading the visual aesthetics of images, this poor quality may also significantly degenerate the performance of many computer vision and multimedia algorithms that are primarily designed for high-quality inputs. In this paper, we propose a simple yet effective low-light image enhancement (LIME) method. More concretely, the illumination of each pixel is first estimated individually by finding the maximum value in R, G, and B channels. Furthermore, we refine the initial illumination map by imposing a structure prior on it, as the final illumination map. Having the well-constructed illumination map, the enhancement can be achieved accordingly. Experiments on a number of challenging low-light images are present to reveal the efficacy of our LIME and show its superiority over several state-of-the-arts in terms of enhancement quality and efficiency.
1,364 citations
"Naturalness Preserved Nonuniform Il..." refers background in this paper
...[17] proposed a weighted L1-norm based regularization to only estimate illumination....
[...]