In this article, we provide a basic introduction to CMOS image-sensor technology, design and performance limits and present recent developments and future directions in this area. We also discuss image-sensor operation and describe the most popular CMOS image-sensor architectures. We note the main non-idealities that limit CMOS image sensor performance, and specify several key performance measures. One of the most important advantages of CMOS image sensors over CCDs is the ability to integrate sensing with analog and digital processing down to the pixel level. Finally, we focus on recent developments and future research directions that are enabled by pixel-level processing, the applications of which promise to further improve CMOS image sensor performance and broaden their applicability beyond current markets.

CMOS image sensors

The biomimetic CMOS dynamic vision and image sensor described in this paper is based on a QVGA (304×240) array of fully autonomous pixels containing event-based change detection and pulse-width-modulation (PWM) imaging circuitry. Exposure measurements are initiated and carried out locally by the individual pixel that has detected a change of brightness in its field-of-view. Pixels do not rely on external timing signals and independently and asynchronously request access to an (asynchronous arbitrated) output channel when they have new grayscale values to communicate. Pixels that are not stimulated visually do not produce output. The visual information acquired from the scene, temporal contrast and grayscale data, are communicated in the form of asynchronous address-events (AER), with the grayscale values being encoded in inter-event intervals. The pixel-autonomous and massively parallel operation ideally results in lossless video compression through complete temporal redundancy suppression at the pixel level. Compression factors depend on scene activity and peak at ~1000 for static scenes. Due to the time-based encoding of the illumination information, very high dynamic range - intra-scene DR of 143 dB static and 125 dB at 30 fps equivalent temporal resolution - is achieved. A novel time-domain correlated double sampling (TCDS) method yields array FPN of 56 dB (9.3 bit) for >10 Lx illuminance.

A QVGA 143 dB Dynamic Range Frame-Free PWM Image Sensor With Lossless Pixel-Level Video Compression and Time-Domain CDS

Temporal noise sets the fundamental limit on image sensor performance, especially under low illumination and in video applications. In a CCD image sensor, temporal noise is primarily due to the photodetector shot noise and the output amplifier thermal and 1/f noise. CMOS image sensors suffer from higher noise than CCDs due to the additional pixel and column amplifier transistor thermal and 1/f noise. Noise analysis is further complicated by the time-varying circuit models, the fact that the reset transistor operates in subthreshold during reset, and the nonlinearity of the charge to voltage conversion, which is becoming more pronounced as CMOS technology scales. The paper presents a detailed and rigorous analysis of temporal noise due to thermal and shot noise sources in CMOS active pixel sensor (APS) that takes into consideration these complicating factors. Performing time-domain analysis, instead of the more traditional frequency-domain analysis, we find that the reset noise power due to thermal noise is at most half of its commonly quoted kT/C value. This result is corroborated by several published experimental data including data presented in this paper. The lower reset noise, however, comes at the expense of image lag. We find that alternative reset methods such as overdriving the reset transistor gate or using a pMOS transistor can alleviate lag, but at the expense of doubling the reset noise power. We propose a new reset method that alleviates lag without increasing reset noise.

/pdf/analysis-of-temporal-noise-in-cmos-photodiode-active-pixel-n3d1gl83sn.pdf

Analysis of temporal noise in CMOS photodiode active pixel sensor

An arbitrated address-event imager has been designed and fabricated in a 0.6-/spl mu/m CMOS process. The imager is composed of 80 /spl times/ 60 pixels of 32 /spl times/ 30 /spl mu/m. The value of the light intensity collected by each photosensitive element is inversely proportional to the pixel's interspike time interval. The readout of each spike is initiated by the individual pixel; therefore, the available output bandwidth is allocated according to pixel output demand. This encoding of light intensities favors brighter pixels, equalizes the number of integrated photons across light intensity, and minimizes power consumption. Tests conducted on the imager showed a large output dynamic range of 180 dB (under bright local illumination) for an individual pixel. The array, on the other hand, produced a dynamic range of 120 dB (under uniform bright illumination and when no lower bound was placed on the update rate per pixel). The dynamic range is 48.9 dB value at 30-pixel updates/s. Power consumption is 3.4 mW in uniform indoor light and a mean event rate of 200 kHz, which updates each pixel 41.6 times per second. The imager is capable of updating each pixel 8.3K times per second (under bright local illumination).

/pdf/a-biomorphic-digital-image-sensor-1jj38n96ux.pdf

A biomorphic digital image sensor

Analysis results demonstrate that multiple sampling can achieve consistently higher signal-to-noise ratio at equal or higher dynamic range than using other image sensor dynamic range enhancement schemes such as well capacity adjusting. Implementing multiple sampling, however, requires much higher readout speeds than can be achieved using typical CMOS active pixel sensor (APS). This paper demonstrates, using a 640/spl times/512 CMOS image sensor with 8-b bit-serial Nyquist rate analog-to-digital converter (ADC) per 4 pixels, that pixel-level ADC enables a highly flexible and efficient implementation of multiple sampling to enhance dynamic range. Since pixel values are available to the ADC's at all times, the number and timing of the samples as well as the number of bits obtained from each sample can be freely selected and read out at fast SRAM speeds. By sampling at exponentially increasing exposure times, pixel values with binary floating-point resolution can be obtained. The 640/spl times/512 sensor is implemented in 0.35-/spl mu/m CMOS technology and achieves 10.5/spl times/10.5 /spl mu/m pixel size at 29% fill factor. Characterization techniques and measured quantum efficiency, sensitivity, ADC transfer curve, and fixed pattern noise are presented. A scene with measured dynamic range exceeding 10000:1 is sampled nine times to obtain an image with dynamic range of 65536:1. Limits on achievable dynamic range using multiple sampling are presented.

/pdf/a-640-spl-times-512-cmos-image-sensor-with-ultra-wide-1uggy7rzfq.pdf

A 640/spl times/512 CMOS image sensor with ultra wide dynamic range floating-point pixel-level ADC

A 256x256 CMOS active pixel sensor (APS) is described for an automotive stereo-vision system. Illumination may vary over several orders of magnitude, requiring a high dynamic range imager. Dynamic range is increased 20/spl times/ using a special clocking scheme for the lateral overflow gate. This dynamic range enhancement offers improvements over previously-described techniques. It simultaneously provides flexibility, user-adjustability, and digital control, with no reduction of fill factor. On-chip column-parallel cyclic analog-to-digital converters (ADCs) produce digital output at frame rates from 30 to 390 frames/s.

D. McGrath

Papers

A 256/spl times/256 CMOS imaging array with wide dynamic range pixels and column-parallel digital output