Obtaining a high quality high dynamic range HDR image in the presence of camera and object movement has been a long-standing challenge. Many methods, known as HDR deghosting algorithms, have been developed over the past ten years to undertake this challenge. Each of these algorithms approaches the deghosting problem from a different perspective, providing solutions with different degrees of complexity, solutions that range from rudimentary heuristics to advanced computer vision techniques. The proposed solutions generally differ in two ways: 1 how to detect ghost regions and 2 what to do to eliminate ghosts. Some algorithms choose to completely discard moving objects giving rise to HDR images which only contain the static regions. Some other algorithms try to find the best image to use for each dynamic region. Yet others try to register moving objects from different images in the spirit of maximizing dynamic range in dynamic regions. Furthermore, each algorithm may introduce different types of artifacts as they aim to eliminate ghosts. These artifacts may come in the form of noise, broken objects, under- and over-exposed regions, and residual ghosting. Given the high volume of studies conducted in this field over the recent years, a comprehensive survey of the state of the art is required. Thus, the first goal of this paper is to provide this survey. Secondly, the large number of algorithms brings about the need to classify them. Thus the second goal of this paper is to propose a taxonomy of deghosting algorithms which can be used to group existing and future algorithms into meaningful classes. Thirdly, the existence of a large number of algorithms brings about the need to evaluate their effectiveness, as each new algorithm claims to outperform its precedents. Therefore, the last goal of this paper is to share the results of a subjective experiment which aims to evaluate various state-of-the-art deghosting algorithms.

The State of the Art in HDR Deghosting: A Survey and Evaluation

We propose a new methodology to acquire HDR video content for autostereoscopic displays by adapting and
augmenting an eight view video camera with standard sensors. To augment the intensity capacity of the sensors,
we combine images taken at dierent exposures. Since the exposure has to be the same for all objectives of our
camera, we x the exposure variation by applying neutral density lters on each objective. Such an approach
has two advantages: several exposures are known for each video frame and we do not need to worry about
synchronization. For each pixel of each view, an HDR value is computed by a weighted average function applied
to the values of matching pixels from all views. The building of the pixel match list is simplied by the property
of our camera which has eight aligned, equally distributed objectives. At each frame, this results in an individual
HDR image for each view while only one exposition per view was taken. The nal eight HDR images are
tone-mapped and interleaved for autostereoscopic display.

High-dynamic range video acquisition with a multiview camera

Creating High Dynamic Range (HDR) images of static scenes is a common procedure nowadays that combinesseveral Low Dynamic Range (LDR) images. However, HDR video and 3D content creation and management isan active, unsolved research topic. This work proposes a method to build HDR images from Low Dynamic Range(LDR)inputimagestakenwithmulti-viewcameras.Weproposeanimageregistrationmethodtoproduce3DHDRcontentforauto-stereoscopicdisplays.ThismethodisbasedonthePatchMatchalgorithmwhichhasbeenadaptedtotakeadvantageofepipolargeometryconstraintsofmulti-viewcameras.Differentimagesimilaritymeasuresareused to improve the accuracy of the matching process. In our case we use an 8-view LDR camera from which wegenerate an 8-view HDR output. Partial results show accurate registration and HDR reconstruction for each LDRinput image.Categories and Subject Descriptors (according to ACM CCS) : I.3.3 [Computer Graphics]: ComputationalPhotography—3D High Dynamic Range1. IntroductionThe creation of high dynamic range (HDR) content has beenan intense ﬁeld of research lately. Several approaches tocombine multi-exposed images either of static or dynamicscenes have been presented with different degrees of suc-cess. In cases where the pixels of source images are notperfectly aligned, ghosting effects may appear in the out-put HDR image. There are two main causes for ghosting:camera movement during acquisition and dynamic objectsin the scene. The ﬁrst one can be easily solved using imagealignment techniques like the bitmap threshold method pre-sented by Ward [War03]. Several methods to deal with dy-namic scenes have been presented but there is not a standardsolution for this, specially when both camera and objectsin the scene move. The lack of robust methods for multi-exposed image registration of dynamic scenes is one of themain drawbacks for HDR video acquisition. The main goalof this work is to propose a method for image registrationof image sequences for auto-stereoscopic 3D HDR contentcreation.Bonnard et al. [BLV 12] have proposed a methodologyto create content that combines depth (3D) and high dy-namic range video for auto-stereoscopic displays. The inputfor such displays is a set of images taken with a ﬁx expo-sure on each objective. The Octo-cam [PcPD 10] has eightobjectives synchronized in time. The different exposures areachieved by mean of neutral density ﬁlters placed in each ob-jective that ﬁx the percentage of light that arrives to each sen-sor. Each ﬁlter divides by two the amount of light received byeach sensor. The fact that the eight sensors are synchronizedprevents of capturing movement in the scene. The ghostingproblem due to dynamic objects does not exist because allthe objectives capture the scene at same time and same shut-ter speed. However, images are different because they aretaken from different points of view and at four different ex-posures like in Figure 1. Images need to be perfectly alignedto avoid artifacts in the construction of an HDR frame foreach view of a video sequence. Bonnard et al. used recon-structed depth information from epipolar geometry to leadthe pixel match procedure. This method lack of robustnessespecially on under and over exposed areas. To avoid this

Patch-based registration for auto-stereoscopic HDR content creation

A device and method of extracting a keypoint, and an image matching system for analyzing an online distribution situation of contents using the same are provided. The device for extracting a keypoint which is robust to image change, includes: a keypoint extraction unit configured to extract a corner region in an image as a keypoint candidate region, and extract the keypoint from the keypoint candidate region; and a keypoint postprocessing unit configured to generate a keypoint descriptor with respect to the extracted keypoint.

Apparatus and method of extracting keypoint and image matching system for analyzing distribution state of contents using the same

High Dynamic Range (HDR) video provides a step change in viewing experience, for example the ability to clearly see the soccer ball when it is kicked from the shadow of the stadium into sunshine. To achieve the full potential of HDR video, so-called true HDR, it is crucial that all the dynamic range that was captured is delivered to the display device and tone mapping is confined only to the display. Furthermore, to ensure widespread uptake of HDR imaging, it should be low cost and available on commodity hardware. This paper describes an end-to-end HDR pipeline for capturing, encoding and streaming high-definition HDR video in real-time using off-the-shelf components. All the lighting that is captured by HDR-enabled consumer cameras is delivered via the pipeline to any display, including HDR displays and even mobile devices with minimum latency. The system thus provides an integrated HDR video pipeline that includes everything from capture to post-production, archival and storage, compression, transmission, and display.

/pdf/live-hdr-video-streaming-on-commodity-hardware-3t4vehsff8.pdf

Live HDR video streaming on commodity hardware

The limited dynamic range of digital images can be extended by composing photographs of the same scene takenwith the same camera at the same view point at di erent exposure times. This is a standard procedure forstatic scenes but a challenging task for dynamic ones. Several methods have been presented but few recover highdynamic range within moving areas. We present a method to recover full high dynamic range (HDR) imagesfrom dynamic scenes, even in moving regions. Our method has 3 steps. Firstly, areas a ected by motion aredetected to generate a ghost mask. Secondly, we register dynamic objects over a reference image (the best exposedimage in the input sequence). Thirdly, we combine the registered input photographs to recover HDR values ina whole image using a weighted average function. Once matching is found, the assembling step guarantees thatall aligned pixels will contribute to the nal result, including dynamic content. Tests were made on more than20 sets of sequences, with moving cars or pedestrians and di erent background. Our results show that ImageMapping Function approach detects best motion regions while Normalized Cross Correlation o ers the best dealspeed-accuracy for image registration. Results from our method o ers better result when moving object areroughly rigid and their movement is mostly rigid. The nal composition is an HDR image with no ghosting andall dynamic content present in HDR values.Keywords: Computational Photography, High Dynamic Range Imaging, Movement Detection, Image Regis-tration

/pdf/full-high-dynamic-range-images-for-dynamic-scenes-4x13vhghto.pdf

P.-P. Vasquez

Papers

Full high-dynamic range images for dynamic scenes