Multi-exposure image fusion (MEF) is emerging as a research hotspot in the fields of image processing and computer vision, which can integrate images with multiple exposure levels into a full exposure image of high quality. It is an economical and effective way to improve the dynamic range of the imaging system and has broad application prospects. In recent years, with the further development of image representation theories such as multi-scale analysis and deep learning, significant progress has been achieved in this field. This paper comprehensively investigates the current research status of MEF methods. The relevant theories and key technologies for constructing MEF models are analyzed and categorized. The representative MEF methods in each category are introduced and summarized. Then, based on the multi-exposure image sequences in static and dynamic scenes, we present a comparative study for 18 representative MEF approaches using nine commonly used objective fusion metrics. Finally, the key issues of current MEF research are discussed, and a development trend for future research is put forward.

/pdf/multi-exposure-image-fusion-techniques-a-comprehensive-29gerp00.pdf

Multi-Exposure Image Fusion Techniques: A Comprehensive Review

The rising demand for high quality display has ensued active research in high dynamic range (HDR) imaging, which has the potential to replace the standard dynamic range imaging. This is due to HDR’s features like accurate reproducibility of a scene with its entire spectrum of visible lighting and color depth. But this capability comes with expensive capture, display, storage and distribution resource requirements. Also, display of HDR images/video content on an ordinary display device with limited dynamic range requires some form of adaptation. Many adaptation algorithms, widely known as tone mapping (TM) operators, have been studied and proposed in the last few decades. In this paper, we present a comprehensive survey of 60 TM algorithms that have been implemented on hardware for acceleration and real-time performance. In this state-of-the-art survey, we will discuss those TM algorithms which have been implemented on GPU [1]–[12], FPGA [13]–[47], and ASIC [48]–[60] in terms of their hardware specifications and performance. Output image quality is an important metric for TM algorithms. From our literature survey we found that, various objective quality metrics have been used to demonstrate the quality of those algorithms hardware implementation. We have compiled those metrics used in this survey [61], [62], and analyzed the relationship between hardware cost, image quality and computational efficiency. Currently, machine learning-based (ML) algorithms have become an important tool to solve many image processing tasks, and this paper concludes with a discussion on the future research directions to realize ML-based TM operators on hardware.

Real-time tone mapping: a survey and cross-implementation hardware benchmark

This paper proposes a novel ghost-free High Dynamic Range (HDR) multi-exposure video acquisition suitable for real-time implementation in embedded systems. While the method is limited to stationary cameras, it achieves, with low requirements on resources, results comparable to state-of-the-art de-ghosting methods that are often very computationally expensive and almost impossible to implement in smart cameras and embedded systems. The paper describes the method itself and includes an evaluation of the performance on selected embedded platforms and a comparison of the results to the state of the art using HDR datasets.

De-ghosted HDR video acquisition for embedded systems

The rising demand for high quality display has ensued active research in high dynamic range (HDR) imaging, which has the potential to replace the standard dynamic range imaging. This is due to HDR's features like accurate reproducibility of a scene with its entire spectrum of visible lighting and color depth. But this capability comes with expensive capture, display, storage and distribution resource requirements. Also, display of HDR images/video content on an ordinary display device with limited dynamic range requires some form of adaptation. Many adaptation algorithms, widely known as tone mapping operators, have been studied and proposed in the last few decades. In this state of the art report, we present a comprehensive survey of 50+ tone mapping algorithms that have been implemented on hardware for acceleration and real-time performance. These algorithms have been adapted or redesigned to make them hardware-friendly. All real-time application poses strict timing constraints which requires time exact processing of the algorithm. This design challenge require novel solution, and in this report we focus on these issues. In this we survey will discuss those tonemap algorithms which have been implemented on GPU [1-10], FPGA [11-41], and ASIC [42-53] in terms of their hardware specifications and performance. Output image quality is an important metric for tonemap algorithms. From our literature survey we found that, various objective quality metrics have been used to demonstrate the functionality of adapting the algorithm on hardware platform. We have compiled and studied all the metrics used in this survey [54-67]. Finally, in this report we demonstrate the link between hardware cost and image quality thereby illustrating the underlying trade-off which will be useful for the research community. 

Real-Time Tone Mapping: A Survey and Cross-Implementation Hardware Benchmark

A real-time high dynamic range intensified complementary metal oxide semiconductor camera based on FPGA

This paper presents a novel FPGA architecture of high dynamic range (HDR) video processing pipeline, based on the capturing of a sequence of differently exposed images. An acquisition process enabling multi-exposure HDR as well as fast implementation of local tone mapping operator involving bilateral filtering is proposed. The HDR acquisition process is enhanced by the application of novel deghosting method, which is dedicated for hardware implementation and proposed in this paper. The hardware processing pipeline is designed with regards to efficiency and performance and the calculations are performed in fixed point arithmetic. The pipeline is suitable for programmable hardware (FPGA—Field Programmable Gate Arrays) implementation and it achieves real-time performance on full HD HDR video which overcomes state-of-the-art solutions that use local tone mapping and deghosting algorithm.

Color HDR video processing architecture for smart camera

HDR video acquisition and processing are quite challenging tasks from the point of both computational complexity and algorithm design. In this paper, we present a real-time HDR processing system evaluated on a custom hardware camera platform. We exploited well known state of the art algorithms for HDR acquisition from sequence of multiexposure frames and real-time tone-mapping. We also present modifications of the algorithms enabling efficient implementation on FPGA platform and real-time performance. The main focus of the paper is on acceleration of Durand local tone-mapping operator involving real-time bilateral filter. The proposed solution is compared to existing research results in terms of speed, resource consumptions, and numerical accuracy.

Svetozar Nosko

Papers

Color HDR video processing architecture for smart camera

De-ghosted HDR video acquisition for embedded systems

True HDR camera with bilateral filter based tone mapping