Images captured under poor illumination conditions often exhibit characteristics such as low brightness, low contrast, a narrow gray range, and color distortion, as well as considerable noise, which seriously affect the subjective visual effect on human eyes and greatly limit the performance of various machine vision systems. The role of low-light image enhancement is to improve the visual effect of such images for the benefit of subsequent processing. This paper reviews the main techniques of low-light image enhancement developed over the past decades. First, we present a new classification of these algorithms, dividing them into seven categories: gray transformation methods, histogram equalization methods, Retinex methods, frequency-domain methods, image fusion methods, defogging model methods and machine learning methods. Then, all the categories of methods, including subcategories, are introduced in accordance with their principles and characteristics. In addition, various quality evaluation methods for enhanced images are detailed, and comparisons of different algorithms are discussed. Finally, the current research progress is summarized, and future research directions are suggested.

/pdf/an-experiment-based-review-of-low-light-image-enhancement-5dhscyutxx.pdf

An Experiment-Based Review of Low-Light Image Enhancement Methods

Nowadays, people are getting used to taking photos to record their daily life, however, the photos are actually not consistent with the real natural scenes. The two main differences are that the photos tend to have low dynamic range (LDR) and low resolution (LR), due to the inherent imaging limitations of cameras. The multi-exposure image fusion (MEF) and image super-resolution (SR) are two widely-used techniques to address these two issues. However, they are usually treated as independent researches. In this paper, we propose a deep Coupled Feedback Network (CF-Net) to achieve MEF and SR simultaneously. Given a pair of extremely over-exposed and under-exposed LDR images with low-resolution, our CF-Net is able to generate an image with both high dynamic range (HDR) and high-resolution. Specifically, the CF-Net is composed of two coupled recursive sub-networks, with LR over-exposed and under-exposed images as inputs, respectively. Each sub-network consists of one feature extraction block (FEB), one super-resolution block (SRB) and several coupled feedback blocks (CFB). The FEB and SRB are to extract high-level features from the input LDR image, which are required to be helpful for resolution enhancement. The CFB is arranged after SRB, and its role is to absorb the learned features from the SRBs of the two sub-networks, so that it can produce a high-resolution HDR image. We have a series of CFBs in order to progressively refine the fused high-resolution HDR image. Extensive experimental results show that our CF-Net drastically outperforms other state-of-the-art methods in terms of both SR accuracy and fusion performance. The software code is available here https://github.com/ytZhang99/CF-Net .

Deep Coupled Feedback Network for Joint Exposure Fusion and Image Super-Resolution

The presence of a wide range of high- and low-resolution devices renders the images to experience changes in respect to aspect ratio and size for better adaptability. This paper proposes a novel and effective technique that vanquishes the problems encountered in the conventional seam carving method. To achieve minimum distortion in salient objects of the image, the proposed technique restricts intersection or overlapping of multiple seams in the horizontal and vertical direction and bypasses them to the neighboring low energy pixel. The proposed technique hinders the pixel selection from a single row or column beyond the defined threshold so as to save the image information and reduce distortion at a single location. To justify the effectiveness of the proposed technique, results have been presented in comparison with the five state-of-the-art image retargeting techniques. Compared with the conventional techniques, this technique shows remarkable results in terms of low distortion percentage. The proposed technique also produces excellent results for shrinkage and enlargement of a single image multiple times.

Structure preservation of image using an efficient content-aware image retargeting technique

The digital elevation model (DEM) is known as one kind of the most significant fundamental geographical data models. The theory, method and application of DEM are hot research issues in geography, especially in geomorphology, hydrology, soil and other related fields. In this paper, we improve the efficient sub-pixel convolutional neural networks (ESPCN) and propose recursive sub-pixel convolutional neural networks (RSPCN) to generate higher-resolution DEMs (HRDEMs) from low-resolution DEMs (LRDEMs). Firstly, the structure of RSPCN is described in detail based on recursion theory. This paper explores the effects of different training datasets, with the self-adaptive learning rate Adam algorithm optimizing the model. Furthermore, the adding-“zero” boundary method is introduced into the RSPCN algorithm as a data preprocessing method, which improves the RSPCN method’s accuracy and convergence. Extensive experiments are conducted to train the method till optimality. Finally, comparisons are made with other traditional interpolation methods, such as bicubic, nearest-neighbor and bilinear methods. The results show that our method has obvious improvements in both accuracy and robustness and further illustrate the feasibility of deep learning methods in the DEM data processing area.

RSPCN: Super-Resolution of Digital Elevation Model Based on Recursive Sub-Pixel Convolutional Neural Networks

Principal Component Analysis is a dimension reduction technique that is widely used in the area of image fusion, classification and face recognition. It cannot be applied on two-dimensional images directly, instead, two-dimensional images must be transformed into one-dimensional vectors prior to applying PCA. Vectorization of images losses the relationships between rows and columns, that may lead to the calculation of misleading principal components. Two-dimensional PCA rectifies this problem by directly dealing with two-dimensional images without prior requirements of vectorization. In this paper, we proposed a novel multi-modal medical image fusion algorithm that is based on two-dimensional PCA. Experiments are conducted to fuse three image-sets of multi-modal images of the brain. Fusion results of proposed algorithms are compared with fusion results of PCA based image fusion algorithm by using seven widely used image quality assessment matrices. The comparison shows the superiority of proposed algorithm over existing PCA based image fusion algorithm.

Multi-modal medical image fusion using 2DPCA

Display of images on different display devices having varied size and aspect ratio requires one to resize them. Many attempts have been made to perform content-aware image retargeting while generating an image compatible with a target display size. Seam carving is one of the image retargeting operators which alters the size of an image by removing least energy pixels. However, it requires high computational time in order to perform retargeting. In this study, the authors accelerate the naive seam carving process by removal or insertion of multiple pixel wide batch seam
 in a single iteration rather than a single pixel wide seam. Along with the energy of pixels to be removed, inserted energy after the removal of a batch seam is also minimised in order to prevent the inclusion of false edges. The width of a batch seam is a critical factor which is made adaptive during the retargeting process to preserve the energy of an image. They have shown a significant decrease in computational time with the increase in the width of a batch seam. They have compared the proposed technique with other state-of-the-art image retargeting operators using different quality assessment metrics and visual results.

Accelerated seam carving for image retargeting

Reflection symmetry aware image retargeting

Image fusion is the process of combining multiple images of a same scene to single high-quality image which has more information than any of the input images. In this paper, we propose a new fusion approach in a spatial domain using propagated image filter. The proposed approach calculates the weight map of every input image using the propagated image filter and gradient domain postprocessing. Propagated image filter exploits cumulative weight construction approach for filtering operation. We show that the proposed approach is able to achieve state-of-the-art results for the problem of multi-exposure fusion for various types of indoor and outdoor natural static scenes with varying amounts of dynamic range.

Multi-exposure Image Fusion Using Propagated Image Filtering

Terrain, representing features of an earth surface, plays a crucial role in many applications such as simulations, route planning, analysis of surface dynamics, computer graphics-based games, entertainment, films, to name a few. With recent advancements in digital technology, these applications demand the presence of high-resolution details in the terrain. In this paper, we propose a novel fully convolutional neural network-based super-resolution architecture to increase the resolution of low-resolution Digital Elevation Model (LRDEM) with the help of information extracted from the corresponding aerial image as a complementary modality. We perform the super-resolution of LRDEM using an attention-based feedback mechanism named 'Attentional Feedback Network' (AFN), which selectively fuses the information from LRDEM and aerial image to enhance and infuse the high-frequency features and to produce the terrain realistically. We compare the proposed architecture with existing state-of-the-art DEM super-resolution methods and show that the proposed architecture outperforms enhancing the resolution of input LRDEM accurately and in a realistic manner.

Diptiben Patel

Papers

Accelerated seam carving for image retargeting

Reflection symmetry aware image retargeting

Multi-exposure Image Fusion Using Propagated Image Filtering

AFN: Attentional Feedback Network based 3D Terrain Super-Resolution

Object occlusion guided stereo image retargeting