A real-time high dynamic range intensified complementary metal oxide semiconductor camera based on FPGA

Due to the recent advances of sensing technologies, high dynamic range (HDR) imaging devices have been developed for the image acquisition of real-world scenes with a wide range of luminance changes. In this article, the hierarchical matching techniques for HDR image pairs are investigated. A bit slicing method is combined with the state-of-the-art stereo matching algorithms for HDR images generated from multiple exposures. Our approach demonstrates that it is possible to use only the most significant 16 bits of the HDR image encoding per pixel to achieve a comparable bad pixel rate on stereo matching. Thus, the computational power and memory constraints can be reduced while benefiting from the HDR images. This makes HDR stereo matching feasible under general resource limited setups. In the experiments, the evaluation and comparison carried out using Middlebury stereo datasets have illustrated the effectiveness of the proposed framework. The source codes are available at: https://github.com/chillwater/HDR_Stereo_Match .

Hierarchical Bit-Plane Slicing for High Dynamic Range Image Stereo Matching

As we all know, CPU/GPU/DSP are the platforms where the most of the image processing in cameras base on, however with the Field Programmable Gate Array (FPGA) shows more advantages there are more and more digital cameras designed with it. In this paper, a wide-spectrum and high-speed CMOS system is proposed using FPGA. Further, DDR3 circuit is also used to cache a full frame information without data loss. At the same time, USB3.0 is used as the data transmission interface in the system. The specific implementation processes on FPGA consist of the CMOS driving module between main board and sensor board as well as a series of video processing pipeline implemented with FPGA. Finally, the system displays a 2048*2046 pixel resolution and has a frame frequency of 22fps, which can detect the wavelength between 200nm and 1100nm with the dynamic range over 90dB. The experimental result shows that the FPGA-based wide-spectrum and high-speed CMOS system can increase the video frame rate and achieve excellent imaging quality.

A wide-spectrum and high-speed CMOS system based on FPGA

Image diagnosis is an important technique in transient process research of high-energy physics. High dynamic range scenes require high linear dynamic range imaging systems. Scientific CMOS (sCMOS) image sensors have widely been used in high-energy physics, nuclear medical imaging, and astronomical observation because of their advantages in the high linear dynamic range. In this paper, we study the gain ratio variation and background value variation of commercial sCMOS image sensors. A self-adaptive fusion method is proposed to realize the fusion of high linear dynamic range images. The proposed method only uses the high gain image and the low gain image of the sCMOS image sensor to evaluate the gain ratio and the background compensation. The measured results show that the error rates of the evaluated gain ratio and background compensation are less than 2% and 6%. Test results show that the self-adaptive fusion method realizes well the fusion effects, which efficiently avoids the influence of gain ratio variation and background value variation.

https://pubs.aip.org/aip/rsi/article-pdf/doi/10.1063/5.0144835/17476199/053705_1_5.0144835.pdf

Self-adaptive fusion method for scientific CMOS image sensors with variable gain ratios and background values.

To overcome the ghosting phenomenon of multi-exposure technology, a new high-dynamic range (HDR) image processing method is proposed in this paper, which combines the features from dual-gain channel images. Further, a complete CMOS camera based on the HDR method is implemented, which produces a real-time HDR live video streams. This camera can capture the details of bright and dark areas in the scene completely with an extended dynamic range up to 95 dB. An ALTERA FPGA is the core processing unit of the entire camera, and it completes all the functional modules of the camera efficiently, including dual-channel video capture, image caching, HDR synthesis and tone mapping. Finally, the real-time HDR video flow has a display resolution of 1920 × 1080 and a frame rate of 60 fps.

Yunsheng Qian

Papers

A high-dynamic range CMOS camera based on dual-gain channels