Imaging in low light is challenging due to low photon count and low SNR. Short-exposure images suffer from noise, while long exposure can induce blur and is often impractical. A variety of denoising, deblurring, and enhancement techniques have been proposed, but their effectiveness is limited in extreme conditions, such as video-rate imaging at night. To support the development of learning-based pipelines for low-light image processing, we introduce a dataset of raw short-exposure low-light images, with corresponding long-exposure reference images. Using the presented dataset, we develop a pipeline for processing low-light images, based on end-to-end training of a fully-convolutional network. The network operates directly on raw sensor data and replaces much of the traditional image processing pipeline, which tends to perform poorly on such data. We report promising results on the new dataset, analyze factors that affect performance, and highlight opportunities for future work.

/pdf/learning-to-see-in-the-dark-4z0eh24hdb.pdf

Learning to See in the Dark

Imaging in low light is challenging due to low photon count and low SNR. Short-exposure images suffer from noise, while long exposure can induce blur and is often impractical. A variety of denoising, deblurring, and enhancement techniques have been proposed, but their effectiveness is limited in extreme conditions, such as video-rate imaging at night. To support the development of learning-based pipelines for low-light image processing, we introduce a dataset of raw short-exposure low-light images, with corresponding long-exposure reference images. Using the presented dataset, we develop a pipeline for processing low-light images, based on end-to-end training of a fully-convolutional network. The network operates directly on raw sensor data and replaces much of the traditional image processing pipeline, which tends to perform poorly on such data. We report promising results on the new dataset, analyze factors that affect performance, and highlight opportunities for future work. The results are shown in the supplementary video at this https URL

We present a deep learning based method for low-light image enhancement. This problem is challenging due to the difficulty in handling various factors simultaneously including brightness, contrast, artifacts and noise. To address this task, we propose the multi-branch low-light enhancement network (MBLLEN). The key idea is to extract rich features up to different levels, so that we can apply enhancement via multiple subnets and finally produce the output image via multi-branch fusion. In this manner, image quality is improved from different aspects. Through extensive experiments, our proposed MBLLEN is found to outperform the state-of-art techniques by a large margin. We additionally show that our method can be directly extended to handle low-light videos.

MBLLEN: Low-Light Image/Video Enhancement Using CNNs.

Preface Prologue: The Exponential Function Chapter 1: Abstract Integration Set-theoretic notations and terminology The concept of measurability Simple functions Elementary properties of measures Arithmetic in [0, ] Integration of positive functions Integration of complex functions The role played by sets of measure zero Exercises Chapter 2: Positive Borel Measures Vector spaces Topological preliminaries The Riesz representation theorem Regularity properties of Borel measures Lebesgue measure Continuity properties of measurable functions Exercises Chapter 3: Lp-Spaces Convex functions and inequalities The Lp-spaces Approximation by continuous functions Exercises Chapter 4: Elementary Hilbert Space Theory Inner products and linear functionals Orthonormal sets Trigonometric series Exercises Chapter 5: Examples of Banach Space Techniques Banach spaces Consequences of Baire's theorem Fourier series of continuous functions Fourier coefficients of L1-functions The Hahn-Banach theorem An abstract approach to the Poisson integral Exercises Chapter 6: Complex Measures Total variation Absolute continuity Consequences of the Radon-Nikodym theorem Bounded linear functionals on Lp The Riesz representation theorem Exercises Chapter 7: Differentiation Derivatives of measures The fundamental theorem of Calculus Differentiable transformations Exercises Chapter 8: Integration on Product Spaces Measurability on cartesian products Product measures The Fubini theorem Completion of product measures Convolutions Distribution functions Exercises Chapter 9: Fourier Transforms Formal properties The inversion theorem The Plancherel theorem The Banach algebra L1 Exercises Chapter 10: Elementary Properties of Holomorphic Functions Complex differentiation Integration over paths The local Cauchy theorem The power series representation The open mapping theorem The global Cauchy theorem The calculus of residues Exercises Chapter 11: Harmonic Functions The Cauchy-Riemann equations The Poisson integral The mean value property Boundary behavior of Poisson integrals Representation theorems Exercises Chapter 12: The Maximum Modulus Principle Introduction The Schwarz lemma The Phragmen-Lindelof method An interpolation theorem A converse of the maximum modulus theorem Exercises Chapter 13: Approximation by Rational Functions Preparation Runge's theorem The Mittag-Leffler theorem Simply connected regions Exercises Chapter 14: Conformal Mapping Preservation of angles Linear fractional transformations Normal families The Riemann mapping theorem The class L Continuity at the boundary Conformal mapping of an annulus Exercises Chapter 15: Zeros of Holomorphic Functions Infinite Products The Weierstrass factorization theorem An interpolation problem Jensen's formula Blaschke products The Muntz-Szas theorem Exercises Chapter 16: Analytic Continuation Regular points and singular points Continuation along curves The monodromy theorem Construction of a modular function The Picard theorem Exercises Chapter 17: Hp-Spaces Subharmonic functions The spaces Hp and N The theorem of F. and M. Riesz Factorization theorems The shift operator Conjugate functions Exercises Chapter 18: Elementary Theory of Banach Algebras Introduction The invertible elements Ideals and homomorphisms Applications Exercises Chapter 19: Holomorphic Fourier Transforms Introduction Two theorems of Paley and Wiener Quasi-analytic classes The Denjoy-Carleman theorem Exercises Chapter 20: Uniform Approximation by Polynomials Introduction Some lemmas Mergelyan's theorem Exercises Appendix: Hausdorff's Maximality Theorem Notes and Comments Bibliography List of Special Symbols Index

Real and Complex Analysis. By W. Rudin. Pp. 412. 84s. 1966. (McGraw-Hill, New York.)

Images captured in low-light conditions usually suffer from very low contrast, which increases the difficulty of subsequent computer vision tasks in a great extent. In this paper, a low-light image enhancement model based on convolutional neural network and Retinex theory is proposed. Firstly, we show that multi-scale Retinex is equivalent to a feedforward convolutional neural network with different Gaussian convolution kernels. Motivated by this fact, we consider a Convolutional Neural Network(MSR-net) that directly learns an end-to-end mapping between dark and bright images. Different fundamentally from existing approaches, low-light image enhancement in this paper is regarded as a machine learning problem. In this model, most of the parameters are optimized by back-propagation, while the parameters of traditional models depend on the artificial setting. Experiments on a number of challenging images reveal the advantages of our method in comparison with other state-of-the-art methods from the qualitative and quantitative perspective.

MSR-net: Low-light Image Enhancement Using Deep Convolutional Network.

This paper presents a low-light image enhancement method using the variational-optimization-based Retinex algorithm. The proposed enhancement method first estimates the initial illumination and uses its gamma corrected version to constrain the illumination component. Next, the variational-based minimization is iteratively performed to separate the reflectance and illumination components. The color assignment of the estimated reflectance component is then performed to restore the color component using the input RGB color channels. Experimental results show that the proposed method can provide better enhanced result without saturation, noise amplification or color distortion.

Low-light image enhancement using variational optimization-based Retinex model

This paper presents a dual autoencoder network model based on the retinex theory to perform the low-light enhancement and noise reduction by combining the stacked and convolutional autoencoders. The proposed method first estimates the spatially smooth illumination component which is brighter than an input low-light image using a stacked autoencoder with a small number of hidden units. Next, we use a convolutional autoencoder which deals with 2-D image information to reduce the amplified noise in the brightness enhancement process. We analyzed and compared roles of the stacked and convolutional autoencoders with the constraint terms of the variational retinex model. In the experiments, we demonstrate the performance of the proposed algorithm by comparing with the state-of-the-art existing low-light and contrast enhancement methods.

Dual Autoencoder Network for Retinex-Based Low-Light Image Enhancement

This paper presents an optimization-based low-light image enhancement method using spatially adaptive �������� -norm based Retinex model. The proposed method adaptively enforces the regularization parameter using the spatially adaptive weight map, which is generated using the bright channel prior (BCP) and local variance map. Since the proposed weight map assigns the smaller weight value at the bright and edge region, the proposed method can perform weak noise reduction to preserve the edges and textures. In addition, the simplified version of the proposed method is presented using the FFT and quantized weight values for the application to consumer devices. Experimental results show that the proposed method can provide better enhanced result without the ι2 -norm minimization artifacts at the low computational cost.

Low-light image enhancement using variational optimization-based retinex model

This paper presents a low-light video restoration algorithm using similar patches among temporally adjacent frames. The proposed artifact-free low-light video restoration algorithm consists of three steps: 1) brightness enhancement using similar patches among temporally adjacent frames and adaptive accumulation; 2) improved color assignment to reduce color distortion; and 3) image fusion for saturation reduction using the guide map. The proposed brightness enhancement step guarantees not to produce any undesired artifacts because of searching the most similar patches among given set of temporally adjacent frames. The color assignment and fusion steps enable a fully automatic color preservation and average brightness control. Experimental results show that the proposed algorithm can better restore high-quality videos without undesired artifacts such as noise amplification, flicker, color distortion, and brightness saturation. As a result, the proposed algorithm can be implemented in a wide range of digital imaging applications such as video surveillance systems and advanced driver assistance systems.

Artifact-Free Low-Light Video Enhancement Using Temporal Similarity and Guide Map

This paper presents a low-light image restoration method based on the variational Retinex model using the bright channel prior (BCP) and total-variation minimization. The proposed method first estimates the bright channel to control the amount of brightness enhancement. Next, the variational Retinex-based energy function is iteratively minimized to estimate the improved illumination and reflectance using the BCP. Contrast of the estimated illumination is enhanced using the gamma correction and histogram equalization to reduce a color distortion and noise amplification. Experimental results show that the proposed method can provide the better restored result than the existing methods without unnatural artifacts such as noise amplification and halo effects near edges.

/pdf/low-light-image-restoration-using-bright-channel-prior-based-28zb9s357r.pdf

Soohwan Yu

Papers

Low-light image enhancement using variational optimization-based Retinex model

Dual Autoencoder Network for Retinex-Based Low-Light Image Enhancement

Low-light image enhancement using variational optimization-based retinex model

Artifact-Free Low-Light Video Enhancement Using Temporal Similarity and Guide Map

Low-light image restoration using bright channel prior-based variational Retinex model