A multiscale retinex for bridging the gap between color images and the human observation of scenes
01 Jul 1997-IEEE Transactions on Image Processing (IEEE Trans Image Process)-Vol. 6, Iss: 7, pp 965-976
TL;DR: This paper extends a previously designed single-scale center/surround retinex to a multiscale version that achieves simultaneous dynamic range compression/color consistency/lightness rendition and defines a method of color restoration that corrects for this deficiency at the cost of a modest dilution in color consistency.
Abstract: Direct observation and recorded color images of the same scenes are often strikingly different because human visual perception computes the conscious representation with vivid color and detail in shadows, and with resistance to spectral shifts in the scene illuminant. A computation for color images that approaches fidelity to scene observation must combine dynamic range compression, color consistency-a computational analog for human vision color constancy-and color and lightness tonal rendition. In this paper, we extend a previously designed single-scale center/surround retinex to a multiscale version that achieves simultaneous dynamic range compression/color consistency/lightness rendition. This extension fails to produce good color rendition for a class of images that contain violations of the gray-world assumption implicit to the theoretical foundation of the retinex. Therefore, we define a method of color restoration that corrects for this deficiency at the cost of a modest dilution in color consistency. Extensive testing of the multiscale retinex with color restoration on several test scenes and over a hundred images did not reveal any pathological behaviour.
Citations
More filters
TL;DR: This work presents a simple and efficient preprocessing chain that eliminates most of the effects of changing illumination while still preserving the essential appearance details that are needed for recognition, and improves robustness by adding Kernel principal component analysis (PCA) feature extraction and incorporating rich local appearance cues from two complementary sources.
Abstract: Making recognition more reliable under uncontrolled lighting conditions is one of the most important challenges for practical face recognition systems. We tackle this by combining the strengths of robust illumination normalization, local texture-based face representations, distance transform based matching, kernel-based feature extraction and multiple feature fusion. Specifically, we make three main contributions: 1) we present a simple and efficient preprocessing chain that eliminates most of the effects of changing illumination while still preserving the essential appearance details that are needed for recognition; 2) we introduce local ternary patterns (LTP), a generalization of the local binary pattern (LBP) local texture descriptor that is more discriminant and less sensitive to noise in uniform regions, and we show that replacing comparisons based on local spatial histograms with a distance transform based similarity metric further improves the performance of LBP/LTP based face recognition; and 3) we further improve robustness by adding Kernel principal component analysis (PCA) feature extraction and incorporating rich local appearance cues from two complementary sources-Gabor wavelets and LBP-showing that the combination is considerably more accurate than either feature set alone. The resulting method provides state-of-the-art performance on three data sets that are widely used for testing recognition under difficult illumination conditions: Extended Yale-B, CAS-PEAL-R1, and Face Recognition Grand Challenge version 2 experiment 4 (FRGC-204). For example, on the challenging FRGC-204 data set it halves the error rate relative to previously published methods, achieving a face verification rate of 88.1% at 0.1% false accept rate. Further experiments show that our preprocessing method outperforms several existing preprocessors for a range of feature sets, data sets and lighting conditions.
2,981 citations
Cites background from "A multiscale retinex for bridging t..."
...Otherwise, extend-as-constant boundary conditions should be used: using extend-as-zero or wrap-around (FFT) boundary conditions significantly reduces the overall performance, in part because it introduces strong gradients at the image borders that disturb the subsequent contrast equalization stage....
[...]
...Hovever, as mentioned above, face recognition is a complex task for which it is useful to include multiple types of features, and we also need to build a final classification stage that can handle residual variability and learn effective models from relatively few training samples....
[...]
07 Jun 2015
TL;DR: This paper proposes an effective feature representation called Local Maximal Occurrence (LOMO), and a subspace and metric learning method called Cross-view Quadratic Discriminant Analysis (XQDA), and presents a practical computation method for XQDA.
Abstract: Person re-identification is an important technique towards automatic search of a person's presence in a surveillance video. Two fundamental problems are critical for person re-identification, feature representation and metric learning. An effective feature representation should be robust to illumination and viewpoint changes, and a discriminant metric should be learned to match various person images. In this paper, we propose an effective feature representation called Local Maximal Occurrence (LOMO), and a subspace and metric learning method called Cross-view Quadratic Discriminant Analysis (XQDA). The LOMO feature analyzes the horizontal occurrence of local features, and maximizes the occurrence to make a stable representation against viewpoint changes. Besides, to handle illumination variations, we apply the Retinex transform and a scale invariant texture operator. To learn a discriminant metric, we propose to learn a discriminant low dimensional subspace by cross-view quadratic discriminant analysis, and simultaneously, a QDA metric is learned on the derived subspace. We also present a practical computation method for XQDA, as well as its regularization. Experiments on four challenging person re-identification databases, VIPeR, QMUL GRID, CUHK Campus, and CUHK03, show that the proposed method improves the state-of-the-art rank-1 identification rates by 2.2%, 4.88%, 28.91%, and 31.55% on the four databases, respectively.
2,209 citations
Cites methods from "A multiscale retinex for bridging t..."
...In this paper, we propose to apply the Retinex algorithm [20, 17, 16] to preprocess person images....
[...]
...We implement the multiscale Retinex algorithm according to [16], which combines the small-scale Retinex for dynamic range compression and the large-scale Retinex for tonal rendition simultaneously....
[...]
TL;DR: A new technique for the display of high-dynamic-range images, which reduces the contrast while preserving detail, is presented, based on a two-scale decomposition of the image into a base layer.
Abstract: We present a new technique for the display of high-dynamic-range images, which reduces the contrast while preserving detail. It is based on a two-scale decomposition of the image into a base layer,...
1,715 citations
Cites methods from "A multiscale retinex for bridging t..."
...Building on previous approaches, our contrast reduction is based on a multiscale decomposition e.g. [Jobson et al. 1997; Pattanaik et al. 1998; Tumblin and Turk 1999]....
[...]
01 Jul 2002
TL;DR: A new technique for the display of high-dynamic-range images, which reduces the contrast while preserving detail, is presented, based on a two-scale decomposition of the image into a base layer, encoding large-scale variations, and a detail layer.
Abstract: We present a new technique for the display of high-dynamic-range images, which reduces the contrast while preserving detail. It is based on a two-scale decomposition of the image into a base layer, encoding large-scale variations, and a detail layer. Only the base layer has its contrast reduced, thereby preserving detail. The base layer is obtained using an edge-preserving filter called the bilateral filter. This is a non-linear filter, where the weight of each pixel is computed using a Gaussian in the spatial domain multiplied by an influence function in the intensity domain that decreases the weight of pixels with large intensity differences. We express bilateral filtering in the framework of robust statistics and show how it relates to anisotropic diffusion. We then accelerate bilateral filtering by using a piecewise-linear approximation in the intensity domain and appropriate subsampling. This results in a speed-up of two orders of magnitude. The method is fast and requires no parameter setting.
1,612 citations
Cites methods from "A multiscale retinex for bridging t..."
...Building on previous approaches, our contrast reduction is based on a multiscale decomposition e.g. [Jobson et al. 1997; Pattanaik et al....
[...]
01 Jul 2002
TL;DR: The results demonstrate that the method is capable of drastic dynamic range compression, while preserving fine details and avoiding common artifacts, such as halos, gradient reversals, or loss of local contrast.
Abstract: We present a new method for rendering high dynamic range images on conventional displays. Our method is conceptually simple, computationally efficient, robust, and easy to use. We manipulate the gradient field of the luminance image by attenuating the magnitudes of large gradients. A new, low dynamic range image is then obtained by solving a Poisson equation on the modified gradient field. Our results demonstrate that the method is capable of drastic dynamic range compression, while preserving fine details and avoiding common artifacts, such as halos, gradient reversals, or loss of local contrast. The method is also able to significantly enhance ordinary images by bringing out detail in dark regions.
1,441 citations
References
More filters
TL;DR: A practical implementation of the retinex is defined without particular concern for its validity as a model for human lightness and color perception, and the trade-off between rendition and dynamic range compression that is governed by the surround space constant is described.
Abstract: The last version of Land's (1986) retinex model for human vision's lightness and color constancy has been implemented and tested in image processing experiments. Previous research has established the mathematical foundations of Land's retinex but has not subjected his lightness theory to extensive image processing experiments. We have sought to define a practical implementation of the retinex without particular concern for its validity as a model for human lightness and color perception. We describe the trade-off between rendition and dynamic range compression that is governed by the surround space constant. Further, unlike previous results, we find that the placement of the logarithmic function is important and produces best results when placed after the surround formation. Also unlike previous results, we find the best rendition for a "canonical" gain/offset applied after the retinex operation. Various functional forms for the retinex surround are evaluated, and a Gaussian form is found to perform better than the inverse square suggested by Land. Images that violate the gray world assumptions (implicit to this retinex) are investigated to provide insight into cases where this retinex fails to produce a good rendition.
1,674 citations
"A multiscale retinex for bridging t..." refers background in this paper
...CONSTRUCTION OF A MULTISCALE CENTER/SURROUND RETINEX The single-scale retinex [10]–[12] is given by (1) where is the retinex output, is the image distribution in the th spectral band, “*” denotes the convolution operation, and is the surround function where is the Gaussian surround space constant, andis selected such that The MSR output is then simply a weighted sum of the outputs of several different SSR outputs....
[...]
...We have previously defined a single-scale retinex [10] (SSR) that can either provide dynamic range compression (small scale), or tonal rendition (large scale), but not both simultaneously....
[...]
...For the SSR this value is 1.5 : 1 or less....
[...]
...The single-scale retinex [10]–[12] is given by...
[...]
...Looking at the forms of the CRF of (5) and the SSR of (1), we conjecture that the CRF represents a spectral analog to the spatial retinex....
[...]
1,314 citations
01 Jan 1995
TL;DR: In this article, the acquisition and use of digital images in a wide variety of scientific fields is discussed. But the focus is on high dynamic range imaging in more than two dimensions.
Abstract: "This guide clearly explains the acquisition and use of digital images in a wide variety of scientific fields. This sixth edition features new sections on selecting a camera with resolution appropriate for use on light microscopes, on the ability of current cameras to capture raw images with high dynamic range, and on imaging in more than two dimensions. It discusses Dmax for X-ray images and combining images with different exposure settings to further extend the dynamic range. This edition also includes a new chapter on shape measurements, a review of new developments in image file searching, and a wide range of new examples and diagrams"
652 citations
TL;DR: The present paper describes a relatively simple alternative technique for the computation of the designator in retinex theory and reports the general operational effectiveness of the new technique, including the competence, not possessed by earlier algorithms, for generating Mach bands.
Abstract: Accepting the first postulate of the retinex theory of color vision that there are three independent lightness-determining mechanisms (one for long waves, one for middle waves, and one for short waves), each operative with less than a millisecond exposure and each served by its own retinal pigment, a basic task of retinex theory becomes the determination of the nature of these mechanisms. Earlier references proposed several workable algorithms. [Land, E. H. (1959) Proc. Natl. Acad. Sci. USA 45, 115-129; Land, E. H. (1959) Proc. Natl. Acad. Sci. USA 45, 636-644; Land, E. H. (1983) Proc. Natl. Acad. Sci. USA 80, 5163-5169; Land, E. H. & McCann, J. J. (1971) J. Opt. Soc. Am. 61, 1-11; Land, E. H. (1986) Vision Res. 26, 7-21.] The present paper describes a relatively simple alternative technique for the computation of the designator in retinex theory and reports the general operational effectiveness of the new technique, including the competence, not possessed by earlier algorithms, for generating Mach bands.
613 citations
"A multiscale retinex for bridging t..." refers methods in this paper
...The idea of the retinex was conceived by Land [2] as a model of the lightness and color perception of human vision....
[...]
...[2] E. Land, “An alternative technique for the computation of the designator in the retinex theory of color vision,” inProc....
[...]
...Through the years, Land evolved the concept from a random walk computation [3] to its last form as a center/surround spatially opponent operation [4], which is related to the neurophysiological functions of individual neurons in the primate retina, lateral geniculate nucleus, and cerebral cortex....
[...]
01 Sep 1996
TL;DR: A multi-scale retinex (MSR) which overcomes this limitation for most scenes and both color rendition and dynamic range compression are successfully accomplished except for some "pathological" scenes that have very strong spectral characteristics in a single band.
Abstract: The retinex is a human perception-based image processing algorithm which provides color constancy and dynamic range compression. We have previously reported on a single-scale retinex (SSR) and shown that it can either achieve color/lightness rendition or dynamic range compression, but not both simultaneously. We now present a multi-scale retinex (MSR) which overcomes this limitation for most scenes. Both color rendition and dynamic range compression are successfully accomplished except for some "pathological" scenes that have very strong spectral characteristics in a single band.
560 citations