Obtaining a high quality high dynamic range HDR image in the presence of camera and object movement has been a long-standing challenge. Many methods, known as HDR deghosting algorithms, have been developed over the past ten years to undertake this challenge. Each of these algorithms approaches the deghosting problem from a different perspective, providing solutions with different degrees of complexity, solutions that range from rudimentary heuristics to advanced computer vision techniques. The proposed solutions generally differ in two ways: 1 how to detect ghost regions and 2 what to do to eliminate ghosts. Some algorithms choose to completely discard moving objects giving rise to HDR images which only contain the static regions. Some other algorithms try to find the best image to use for each dynamic region. Yet others try to register moving objects from different images in the spirit of maximizing dynamic range in dynamic regions. Furthermore, each algorithm may introduce different types of artifacts as they aim to eliminate ghosts. These artifacts may come in the form of noise, broken objects, under- and over-exposed regions, and residual ghosting. Given the high volume of studies conducted in this field over the recent years, a comprehensive survey of the state of the art is required. Thus, the first goal of this paper is to provide this survey. Secondly, the large number of algorithms brings about the need to classify them. Thus the second goal of this paper is to propose a taxonomy of deghosting algorithms which can be used to group existing and future algorithms into meaningful classes. Thirdly, the existence of a large number of algorithms brings about the need to evaluate their effectiveness, as each new algorithm claims to outperform its precedents. Therefore, the last goal of this paper is to share the results of a subjective experiment which aims to evaluate various state-of-the-art deghosting algorithms.

The State of the Art in HDR Deghosting: A Survey and Evaluation

Exemplary methods, apparatuses, and systems for image processing are described. One or more reference images are selected based on image quality scores. At least a portion of each reference image is merged to create an output image. An output image with motion artifacts is compared to a target to correct the motion artifacts of the output image.

Reference image selection for motion ghost filtering

Apparatuses and methods for reading a set of images to merge together into a high dynamic range (HDR) output image are described. Images have a respective HDR weight and a respective ghost-free weight. Images are merged together using the weighted average of the set of input images using the ghost-free weight. A difference image is determined based on a difference between each pixel within a HDR output image and each respective pixel within a reference image used to create the HDR output image.

Generation of ghost-free high dynamic range images

Good image quality of the wireless capsule endoscopy (WCE) is the key for doctors to diagnose gastrointestinal (GI) tract diseases. However, the poor illumination, limited performance of the camera in WCE, and complex environment in the GI tract usually result in low-quality endoscopic images. Existing image enhancement methods only use the information of the image itself or multiple images of the same scene to accomplish the enhancement. In this paper, we propose an adaptive image enhancement method based on guide image and fraction-power transformation. First, intensities of endoscopic images are analyzed to assess the illumination conditions. Second, images captured under poor illumination conditions are enhanced by a brand-new image enhancement method called adaptive guide image based enhancement (AGIE). AGIE enhances low-quality images by using the information of a good quality image of the similar scene. Otherwise, images are enhanced by the proposed adaptive fraction-power transformation. Experimental results show that the proposed method improves the average intensity of endoscopic images by 64.20% and the average local entropy by 31.25%, which outperforms the state-of-art methods.

Adaptive Image Enhancement Based on Guide Image and Fraction-Power Transformation for Wireless Capsule Endoscopy


 Computer simulation can be a useful tool when designing robots expected to operate independently in unstructured environments. In this context, one needs to simulate the dynamics of the robot’s mechanical system, the environment in which the robot operates, and the sensors which facilitate the robot’s perception of the environment. Herein, we focus on the sensing simulation task by presenting a virtual sensing framework built alongside an open-source, multi-physics simulation platform called Chrono. This framework supports camera, lidar, GPS, and IMU simulation. We discuss their modeling as well as the noise and distortion implemented to increase the realism of the synthetic sensor data. We close with two examples that show the sensing simulation framework at work: one pertains to a reduced scale autonomous vehicle and the second is related to a vehicle driven in a digital replica of a Madison neighborhood.

A Sensor Simulation Framework for Training and Testing Robots and Autonomous Vehicles

We propose a simple ghost-free exposure fusion algorithm in which three (over-, mid-, under-) exposures are input for fast embedded HDR imaging. Previous methods in this area require a complicated post-processing, such as morphological operations and large size kernel operations. To remove ghost artifacts successfully even with a low computing power, first of all an over-exposure input is set as a reference image, and non-ghostness probability is applied to acquire a ghost-free fusion weight. Through some experimental analysis, we show that such a reference image setting scheme makes it possible to skip the heavily time consuming post-processing.

A simple ghost-free exposure fusion for embedded HDR imaging

This paper presents a new low-light multi-frame fusion algorithm to get a bright and clear shot even under dark conditions. To this end, using multiple short-exposure images and one proper-exposure blurry image as an input, a new hierarchical block-wise temporal noise filtering is done. Finally, an online image restoration of the denoising result is conducted along with the blurry image input. Test results on real low-light scene show its effectiveness like fast processing speed and satisfactory visual quality.

A fast low-light multi-image fusion with online image restoration

A method and apparatus for processing numeric calculation are provided. The method includes determining a shift bit and an index bit that falls within an index range of a lookup table from among bits representing a divisor scaled up by an offset, obtaining a replacement value corresponding to an index value of the determined index bit by using the lookup table, multiplying a dividend scaled up by the offset by the obtained replacement value, and outputting a value corresponding to a division operation by correcting a scale of a result of the multiplication using a right shift operation.

Apparatus and method of processing numeric calculation

An image signal processor for a CMOS image sensor consists of many complicated functions. In this paper, the full chain of camera ISP functions for smart devices is presented. The every function in the chain is fully converted to fixed point arithmetic and no special function is used for easy porting to reconfigurable processors. To verify the performance of the proposed ISP chain, series of test are performed, and all measured values satisfied the quality requirements.

Implementation of an image signal processor for reconfigurable processors

An image-processing method of removing noise from an image includes: obtaining standard representative values for standard regions of a standard image and reference representative values for reference regions of a reference image, and obtaining a plurality of hierarchical images having different resolutions with respect to the standard image and the reference image based on the standard representative values and the reference representative values; obtaining the standard representative values for the standard regions of the standard image and the reference representative values for the reference regions of the reference image from the hierarchical images; determining a similarity between the standard regions and the reference regions from the standard representative values and the reference representative values; and removing noise by applying filtering to the standard image and the reference image depending on the determined similarity.

Yong-Min Tai

Papers

A simple ghost-free exposure fusion for embedded HDR imaging

A fast low-light multi-image fusion with online image restoration

Apparatus and method of processing numeric calculation

Implementation of an image signal processor for reconfigurable processors

Method and apparatus for image processing