Obtaining a high quality high dynamic range HDR image in the presence of camera and object movement has been a long-standing challenge. Many methods, known as HDR deghosting algorithms, have been developed over the past ten years to undertake this challenge. Each of these algorithms approaches the deghosting problem from a different perspective, providing solutions with different degrees of complexity, solutions that range from rudimentary heuristics to advanced computer vision techniques. The proposed solutions generally differ in two ways: 1 how to detect ghost regions and 2 what to do to eliminate ghosts. Some algorithms choose to completely discard moving objects giving rise to HDR images which only contain the static regions. Some other algorithms try to find the best image to use for each dynamic region. Yet others try to register moving objects from different images in the spirit of maximizing dynamic range in dynamic regions. Furthermore, each algorithm may introduce different types of artifacts as they aim to eliminate ghosts. These artifacts may come in the form of noise, broken objects, under- and over-exposed regions, and residual ghosting. Given the high volume of studies conducted in this field over the recent years, a comprehensive survey of the state of the art is required. Thus, the first goal of this paper is to provide this survey. Secondly, the large number of algorithms brings about the need to classify them. Thus the second goal of this paper is to propose a taxonomy of deghosting algorithms which can be used to group existing and future algorithms into meaningful classes. Thirdly, the existence of a large number of algorithms brings about the need to evaluate their effectiveness, as each new algorithm claims to outperform its precedents. Therefore, the last goal of this paper is to share the results of a subjective experiment which aims to evaluate various state-of-the-art deghosting algorithms.

The State of the Art in HDR Deghosting: A Survey and Evaluation

We propose a new methodology to acquire HDR video content for autostereoscopic displays by adapting and
augmenting an eight view video camera with standard sensors. To augment the intensity capacity of the sensors,
we combine images taken at dierent exposures. Since the exposure has to be the same for all objectives of our
camera, we x the exposure variation by applying neutral density lters on each objective. Such an approach
has two advantages: several exposures are known for each video frame and we do not need to worry about
synchronization. For each pixel of each view, an HDR value is computed by a weighted average function applied
to the values of matching pixels from all views. The building of the pixel match list is simplied by the property
of our camera which has eight aligned, equally distributed objectives. At each frame, this results in an individual
HDR image for each view while only one exposition per view was taken. The nal eight HDR images are
tone-mapped and interleaved for autostereoscopic display.

High-dynamic range video acquisition with a multiview camera

It is possible to generate stereo high dynamic range (HDR) images/videos by using a pair of cameras with different exposure parameters. In this article, a learning-based stereo HDR imaging (SHDRI) method with three modules is proposed. In the proposed method, we construct three convolutional neural network (CNN) modules that perform specific tasks, including exposure calibration CNN (EC-CNN) module, hole-filling CNN (HF-CNN) module and HDR fusion CNN (HDRF-CNN) module, to combine with traditional image processing methods to model SHDRI pipeline. To avoid ambiguity, we assume that the left-view image is under-exposed and the right-view image is over-exposed. Specifically, the EC-CNN module is first constructed to convert stereo multi-exposure images into the same exposure to facilitate subsequent stereo matching. Then, based on the estimated disparity map, the right-view image is forward-warped to generate the initial left-view over-exposure image. After that, extra exposure information is utilized to guide hole-filling. Finally, the HDRF-CNN module is constructed and employed to extract fusion features to fuse the hole-filled left-view over-exposure image with the original left-view under-exposure image into the left-view HDR image. Right-view HDR images can be generated in the same way. In addition, we propose an effective two-phase training strategy to overcome the lack of a sufficient large stereo multi-exposure dataset. The experimental results demonstrate that the proposed method can generate stereo HDR images with high visual quality. Furthermore, the proposed method achieves better performance in comparison with the latest SHDRI method.

Learning Stereo High Dynamic Range Imaging From A Pair of Cameras With Different Exposure Parameters

A laser based structured-light technique is proposed for use in three-dimensional reconstruction. This technique is produced from an extensive experimental evaluation for the fundamental laser stripe extraction methods used with the structured-laser-light technique. Based on this evaluation, the proposed technique uses the extreme with centroid for calculating the probable laser stripe’s locations. It then extracts the global visual information from the scanned scene and uses this information to filter out all the unreliable locations. This makes the new technique has superior extraction capability for the structures-laser-light and robust to the scanning conditions that disturb the reconstruction process. The performance of the proposed technique was evaluated through a series of experiments. The evaluation results show high 3D reconstruction performance for the proposed technique in comparison to the traditional technique when extracting laser-stripes. They also show how the global visual information is efficient for locating these stripes from the other illumination sources.

Laser-based structured light technique for 3D reconstruction using Extreme Laser stripes extraction method with global information extraction

Creating High Dynamic Range (HDR) images of static scenes is a common procedure nowadays that combinesseveral Low Dynamic Range (LDR) images. However, HDR video and 3D content creation and management isan active, unsolved research topic. This work proposes a method to build HDR images from Low Dynamic Range(LDR)inputimagestakenwithmulti-viewcameras.Weproposeanimageregistrationmethodtoproduce3DHDRcontentforauto-stereoscopicdisplays.ThismethodisbasedonthePatchMatchalgorithmwhichhasbeenadaptedtotakeadvantageofepipolargeometryconstraintsofmulti-viewcameras.Differentimagesimilaritymeasuresareused to improve the accuracy of the matching process. In our case we use an 8-view LDR camera from which wegenerate an 8-view HDR output. Partial results show accurate registration and HDR reconstruction for each LDRinput image.Categories and Subject Descriptors (according to ACM CCS) : I.3.3 [Computer Graphics]: ComputationalPhotography—3D High Dynamic Range1. IntroductionThe creation of high dynamic range (HDR) content has beenan intense ﬁeld of research lately. Several approaches tocombine multi-exposed images either of static or dynamicscenes have been presented with different degrees of suc-cess. In cases where the pixels of source images are notperfectly aligned, ghosting effects may appear in the out-put HDR image. There are two main causes for ghosting:camera movement during acquisition and dynamic objectsin the scene. The ﬁrst one can be easily solved using imagealignment techniques like the bitmap threshold method pre-sented by Ward [War03]. Several methods to deal with dy-namic scenes have been presented but there is not a standardsolution for this, specially when both camera and objectsin the scene move. The lack of robust methods for multi-exposed image registration of dynamic scenes is one of themain drawbacks for HDR video acquisition. The main goalof this work is to propose a method for image registrationof image sequences for auto-stereoscopic 3D HDR contentcreation.Bonnard et al. [BLV 12] have proposed a methodologyto create content that combines depth (3D) and high dy-namic range video for auto-stereoscopic displays. The inputfor such displays is a set of images taken with a ﬁx expo-sure on each objective. The Octo-cam [PcPD 10] has eightobjectives synchronized in time. The different exposures areachieved by mean of neutral density ﬁlters placed in each ob-jective that ﬁx the percentage of light that arrives to each sen-sor. Each ﬁlter divides by two the amount of light received byeach sensor. The fact that the eight sensors are synchronizedprevents of capturing movement in the scene. The ghostingproblem due to dynamic objects does not exist because allthe objectives capture the scene at same time and same shut-ter speed. However, images are different because they aretaken from different points of view and at four different ex-posures like in Figure 1. Images need to be perfectly alignedto avoid artifacts in the construction of an HDR frame foreach view of a video sequence. Bonnard et al. used recon-structed depth information from epipolar geometry to leadthe pixel match procedure. This method lack of robustnessespecially on under and over exposed areas. To avoid this

Patch-based registration for auto-stereoscopic HDR content creation

The limited dynamic range of digital images can be extended by composing photographs of the same scene takenwith the same camera at the same view point at di erent exposure times. This is a standard procedure forstatic scenes but a challenging task for dynamic ones. Several methods have been presented but few recover highdynamic range within moving areas. We present a method to recover full high dynamic range (HDR) imagesfrom dynamic scenes, even in moving regions. Our method has 3 steps. Firstly, areas a ected by motion aredetected to generate a ghost mask. Secondly, we register dynamic objects over a reference image (the best exposedimage in the input sequence). Thirdly, we combine the registered input photographs to recover HDR values ina whole image using a weighted average function. Once matching is found, the assembling step guarantees thatall aligned pixels will contribute to the nal result, including dynamic content. Tests were made on more than20 sets of sequences, with moving cars or pedestrians and di erent background. Our results show that ImageMapping Function approach detects best motion regions while Normalized Cross Correlation o ers the best dealspeed-accuracy for image registration. Results from our method o ers better result when moving object areroughly rigid and their movement is mostly rigid. The nal composition is an HDR image with no ghosting andall dynamic content present in HDR values.Keywords: Computational Photography, High Dynamic Range Imaging, Movement Detection, Image Regis-tration

/pdf/full-high-dynamic-range-images-for-dynamic-scenes-4x13vhghto.pdf

Full high-dynamic range images for dynamic scenes

Multiscopic HDR Image sequence generation.

High dynamic range (HDR) imaging and stereoscopic 3D are active research fields, with challenges posed in various directions: acquisition, processing, and display. This chapter analyzes the latest advances in stereo HDR imaging acquisition from multiple exposures, in order to highlight the current progress toward a common target: stereoscopic 3D HDR video. A method to build multiscopic HDR images from low dynamic range (LDR) multiexposures is proposed. This method is based on a patch-match algorithm which has been adapted and improved to take advantage of epipolar geometry constraints of stereo images. Experimental results show accurate matching between stereo images that allow generating HDR for each LDR view of the sequence.

HDR Multiview Image Sequence Generation: Toward 3D HDR Video

The convergence between high dynamic range (HDR) images/video and stereoscopic/3D images/video is an active research field, with limitations coming from various aspects: acquisition, processing, and display. This work analyzes the latest advances in HDR video and stereo HDR video acquisition from multiple exposures in order to highlight the current progress towards a common target: 3D HDR video. The most relevant existing techniques are presented and discussed.

Raissel Ramirez Orozco

Papers

Full high-dynamic range images for dynamic scenes

Patch-based registration for auto-stereoscopic HDR content creation

Multiscopic HDR Image sequence generation.

HDR Multiview Image Sequence Generation: Toward 3D HDR Video

Multiview HDR Video Sequence Generation