We present a new technique for the display of high-dynamic-range images, which reduces the contrast while preserving detail. It is based on a two-scale decomposition of the image into a base layer, encoding large-scale variations, and a detail layer. Only the base layer has its contrast reduced, thereby preserving detail. The base layer is obtained using an edge-preserving filter called the bilateral filter. This is a non-linear filter, where the weight of each pixel is computed using a Gaussian in the spatial domain multiplied by an influence function in the intensity domain that decreases the weight of pixels with large intensity differences. We express bilateral filtering in the framework of robust statistics and show how it relates to anisotropic diffusion. We then accelerate bilateral filtering by using a piecewise-linear approximation in the intensity domain and appropriate subsampling. This results in a speed-up of two orders of magnitude. The method is fast and requires no parameter setting.

/pdf/fast-bilateral-filtering-for-the-display-of-high-dynamic-fprymiweg7.pdf

Fast bilateral filtering for the display of high-dynamic-range images

This tutorial reviews image alignment and image stitching algorithms. Image alignment algorithms can discover the correspondence relationships among images with varying degrees of overlap. They are ideally suited for applications such as video stabilization, summarization, and the creation of panoramic mosaics. Image stitching algorithms take the alignment estimates produced by such registration algorithms and blend the images in a seamless manner, taking care to deal with potential problems such as blurring or ghosting caused by parallax and scene movement as well as varying image exposures. This tutorial reviews the basic motion models underlying alignment and stitching algorithms, describes effective direct (pixel-based) and feature-based alignment algorithms, and describes blending algorithms used to produce seamless mosaics. It ends with a discussion of open research problems in the area.

/pdf/image-alignment-and-stitching-a-tutorial-57yf6bt9xq.pdf

Image Alignment and Stitching: A Tutorial

In this article, we provide a basic introduction to CMOS image-sensor technology, design and performance limits and present recent developments and future directions in this area. We also discuss image-sensor operation and describe the most popular CMOS image-sensor architectures. We note the main non-idealities that limit CMOS image sensor performance, and specify several key performance measures. One of the most important advantages of CMOS image sensors over CCDs is the ability to integrate sensing with analog and digital processing down to the pixel level. Finally, we focus on recent developments and future research directions that are enabled by pixel-level processing, the applications of which promise to further improve CMOS image sensor performance and broaden their applicability beyond current markets.

CMOS image sensors

The biomimetic CMOS dynamic vision and image sensor described in this paper is based on a QVGA (304×240) array of fully autonomous pixels containing event-based change detection and pulse-width-modulation (PWM) imaging circuitry. Exposure measurements are initiated and carried out locally by the individual pixel that has detected a change of brightness in its field-of-view. Pixels do not rely on external timing signals and independently and asynchronously request access to an (asynchronous arbitrated) output channel when they have new grayscale values to communicate. Pixels that are not stimulated visually do not produce output. The visual information acquired from the scene, temporal contrast and grayscale data, are communicated in the form of asynchronous address-events (AER), with the grayscale values being encoded in inter-event intervals. The pixel-autonomous and massively parallel operation ideally results in lossless video compression through complete temporal redundancy suppression at the pixel level. Compression factors depend on scene activity and peak at ~1000 for static scenes. Due to the time-based encoding of the illumination information, very high dynamic range - intra-scene DR of 143 dB static and 125 dB at 30 fps equivalent temporal resolution - is achieved. A novel time-domain correlated double sampling (TCDS) method yields array FPN of 56 dB (9.3 bit) for >10 Lx illuminance.

A QVGA 143 dB Dynamic Range Frame-Free PWM Image Sensor With Lossless Pixel-Level Video Compression and Time-Domain CDS

http://eia.udg.es/%7Eqsalvi/papers/2006-MJ.pdf

Review of CMOS image sensors

Temporal noise sets the fundamental limit on image sensor performance, especially under low illumination and in video applications. In a CCD image sensor, temporal noise is primarily due to the photodetector shot noise and the output amplifier thermal and 1/f noise. CMOS image sensors suffer from higher noise than CCDs due to the additional pixel and column amplifier transistor thermal and 1/f noise. Noise analysis is further complicated by the time-varying circuit models, the fact that the reset transistor operates in subthreshold during reset, and the nonlinearity of the charge to voltage conversion, which is becoming more pronounced as CMOS technology scales. The paper presents a detailed and rigorous analysis of temporal noise due to thermal and shot noise sources in CMOS active pixel sensor (APS) that takes into consideration these complicating factors. Performing time-domain analysis, instead of the more traditional frequency-domain analysis, we find that the reset noise power due to thermal noise is at most half of its commonly quoted kT/C value. This result is corroborated by several published experimental data including data presented in this paper. The lower reset noise, however, comes at the expense of image lag. We find that alternative reset methods such as overdriving the reset transistor gate or using a pMOS transistor can alleviate lag, but at the expense of doubling the reset noise power. We propose a new reset method that alleviates lag without increasing reset noise.

/pdf/analysis-of-temporal-noise-in-cmos-photodiode-active-pixel-n3d1gl83sn.pdf

Analysis of temporal noise in CMOS photodiode active pixel sensor

Analysis results demonstrate that multiple sampling can achieve consistently higher signal-to-noise ratio at equal or higher dynamic range than using other image sensor dynamic range enhancement schemes such as well capacity adjusting. Implementing multiple sampling, however, requires much higher readout speeds than can be achieved using typical CMOS active pixel sensor (APS). This paper demonstrates, using a 640/spl times/512 CMOS image sensor with 8-b bit-serial Nyquist rate analog-to-digital converter (ADC) per 4 pixels, that pixel-level ADC enables a highly flexible and efficient implementation of multiple sampling to enhance dynamic range. Since pixel values are available to the ADC's at all times, the number and timing of the samples as well as the number of bits obtained from each sample can be freely selected and read out at fast SRAM speeds. By sampling at exponentially increasing exposure times, pixel values with binary floating-point resolution can be obtained. The 640/spl times/512 sensor is implemented in 0.35-/spl mu/m CMOS technology and achieves 10.5/spl times/10.5 /spl mu/m pixel size at 29% fill factor. Characterization techniques and measured quantum efficiency, sensitivity, ADC transfer curve, and fixed pattern noise are presented. A scene with measured dynamic range exceeding 10000:1 is sampled nine times to obtain an image with dynamic range of 65536:1. Limits on achievable dynamic range using multiple sampling are presented.

/pdf/a-640-spl-times-512-cmos-image-sensor-with-ultra-wide-1uggy7rzfq.pdf

A 640/spl times/512 CMOS image sensor with ultra wide dynamic range floating-point pixel-level ADC

Analysis of 1/f noise in MOSFET circuits is typically performed in the frequency domain using the standard stationary 1/f noise model. Recent experimental results, however, have shown that the estimates using this model can be quite inaccurate especially for switched circuits. In the case of a periodically switched transistor, measured 1/f noise power spectral density (psd) was shown to be significantly lower than the estimate using the standard 1/f noise model. For a ring oscillator, measured 1/f-induced phase noise psd was shown to be significantly lower than the estimate using the standard 1/f noise model. For a source follower reset circuit, measured 1/f noise power was also shown to be lower than the estimate using the standard 1/f model. In analyzing noise in the follower reset circuit using frequency-domain analysis, a low cutoff frequency that is inversely proportional to the circuit on-time is assumed. The choice of this low cutoff frequency is quite arbitrary and can cause significant inaccuracy in estimating noise power. Moreover, during reset, the circuit is not in steady state, and thus frequency-domain analysis does not apply. This paper proposes a nonstationary extension of the standard 1/f noise model, which allows us to analyze 1/f noise in switched MOSFET circuits more accurately. Using our model, we analyze noise for the three aforementioned switched circuit examples and obtain results that are consistent with the reported measurements.

/pdf/analysis-of-1-f-noise-in-switched-mosfet-circuits-y0yiikbwt7.pdf

Analysis of 1/f noise in switched MOSFET circuits

The standard method for measuring QE for a CCD sensor is not adequate for CMOS APS since it does not take into consideration the random offset, gain variations, and nonlinearity introduced by the APS readout circuits. The paper presents a new method to accurately estimate QE of an APS. Instead of varying illumination as in the CCD method, illumination is kept constant and the pixel output is continuously observed - sampling at regular intervals. This makes it possible to eliminate random offset. The experiment is repeated multiple times to obtain good estimates of the pixel output mean and variance at each sample time. The sensor response is approximated by a piecewise linear function and using the Poisson statistics of shot noise gain, charge and read noise are estimated for each line segment. This procedure is repeated at no illumination so that dark charge may be estimated and subtracted from the total charge estimates. The method can also be used to estimate readout noise and gain FPN. Results from 64 X 64 pixel APS test structures implemented in a 0.35 micrometers CMOS process are reported. Using 6 different chips and 16 pixels per chip QE equals 0.37, gain FPN equals 2 percent, dark charge equals 832e-, and readout noise equals 40e-, are estimated.

/pdf/method-for-estimating-quantum-efficiency-for-cmos-image-363cmak5gl.pdf

Method for estimating quantum efficiency for CMOS image sensors

Temporal noise sets a fundamental limit on image sensor performance, especially under low illumination and in video applications. In a CCD image sensor, temporal noise is well studied and characterized. It is primarily due to the photodetector shot noise and the thermal and 1/f noise of the output charge to voltage amplifier. In a CMOS APS several addition sources contribute to temporal noise, including the noise due to the pixel rest, follower, and access transistors. The analysis of noise is further complicated by the nonlinearity of the APS charge to voltage characteristics, which is becoming more pronounced as CMOS technology scales, and the fact that the reset transistor operates below threshold for most of the reset time. The paper presents an accurate analysis of temporal noise in APS. We analyze the noise for each stage of the sensor operation, and identify the noise contribution from each source. We analyze noise due to photodetector shot noise taking nonlinearity into consideration. We find that nonlinearity improves SNR reset transistor shot noise is at most half the commonly quoted value. Using HSPICE simulation, we find the noise due to the follower and access transistors. As expected we find that at low illumination reset noise dominates, while at high illumination photodetector shot noise dominates. Finally, we present experimental results from test structures fabricated in 0.35(mu) CMOS processes. We find that both measured peak SNR and reset noise values match well with the results of our analysis.© (1999) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

/pdf/analysis-of-temporal-noise-in-cmos-aps-3i28sfw1wz.pdf

Hui Tian

Papers

Analysis of temporal noise in CMOS photodiode active pixel sensor

A 640/spl times/512 CMOS image sensor with ultra wide dynamic range floating-point pixel-level ADC

Analysis of 1/f noise in switched MOSFET circuits

Method for estimating quantum efficiency for CMOS image sensors

Analysis of temporal noise in CMOS APS