A CMOS image sensor architecture with built-in single-shot compressed sensing is described. The image sensor employs a conventional 4-T pixel and per-column ΣΔ ADCs. The compressed sensing measurements are obtained via a column multiplexer that sequentially applies randomly selected pixel values to the input of each ΣΔ modulator. At the end of readout, each ADC outputs a quantized value of the average of the pixel values applied to its input. The image is recovered from the random linear measurements off-chip using numerical optimization algorithms. To demonstrate this architecture, a 256x256 pixel CMOS image sensor is fabricated in 0.15 μm CIS process. The sensor can operate in compressed sensing mode with compression ratio 1/4, 1/8, or 1/16 at 480, 960, or 1920 fps, respectively, or in normal capture mode with no compressed sensing at a maximum frame rate of 120 fps. Measurement results demonstrate capture in compressed sensing mode at roughly the same readout noise of 351 μVrms and power consumption of 96.2 mW of normal capture at 120 fps. This performance is achieved with only 1.8% die area overhead. Image reconstruction shows modest quality loss relative to normal capture and significantly higher image quality than downsampling.

/pdf/cmos-image-sensor-with-per-column-sd-adc-and-programmable-4fjz1c6coh.pdf

CMOS Image Sensor With Per-Column ΣΔ ADC and Programmable Compressed Sensing

We introduce a 512 × 424 time-of-flight (TOF) depth image sensor designed in a TSMC 0.13 μm LP 1P5M CMOS process, suitable for use in Microsoft Kinect for XBOX ONE. The 10 μm × 10 μm pixel incorporates a TOF detector that operates using the quantum efficiency modulation (QEM) technique at high modulation frequencies of up to 130 MHz, achieves a modulation contrast of 67% at 50 MHz and a responsivity of 0.14 A/W at 860 nm. The TOF sensor includes a 2 GS/s 10 bit signal path, which is used for the high ADC bandwidth requirements of the system that requires many ADC conversions per frame. The chip also comprises a clock generation circuit featuring a programmable phase and frequency clock generator with 312.5-ps phase step resolution derived from a 1.6 GHz oscillator. An integrated shutter engine and a programmable digital micro-sequencer allows an extremely flexible multi-gain/multi-shutter and multi-frequency/multi-phase operation. All chip data is transferred using two 4-lane MIPI D-PHY interfaces with a total of 8 Gb/s input/output bandwidth. The reported experimental results demonstrate a wide depth range of operation (0.8–4.2 m), small accuracy error ( $ 1%), very low depth uncertainty ( $ 0.5% of actual distance), and very high dynamic range ( $>$ 64 dB).

A 0.13 μm CMOS System-on-Chip for a 512 × 424 Time-of-Flight Image Sensor With Multi-Frequency Photo-Demodulation up to 130 MHz and 2 GS/s ADC

A major hurdle to frequently performing mobile computer vision tasks is the high power consumption of image sensing. In this work, we report the first publicly known experimental and analytical characterization of CMOS image sensors. We find that modern image sensors are not energy-proportional: energy per pixel is in fact inversely proportional to frame rate and resolution of image capture, and thus image sensor systems fail to provide an important principle of energy-aware system design: trading quality for energy efficiency. We reveal two energy-proportional mechanisms, supported by current image sensors but unused by mobile systems: (i) using an optimal clock frequency reduces the power up to 50% or 30% for low-quality single frame (photo) and sequential frame (video) capturing, respectively; (ii) by entering low-power standby mode between frames, an image sensor achieves almost constant energy per pixel for video capture at low frame rates, resulting in an additional 40% power reduction. We also propose architectural modifications to the image sensor that would further improve operational efficiency. Finally, we use computer vision benchmarks to show the performance and efficiency tradeoffs that can be achieved with existing image sensors. For image registration, a key primitive for image mosaicking and depth estimation, we can achieve a 96% success rate at 3 FPS and 0.1 MP resolution. At these quality metrics, an optimal clock frequency reduces image sensor power consumption by 36% and aggressive standby mode reduces power consumption by 95%.

/pdf/energy-characterization-and-optimization-of-image-sensing-1joepk5d31.pdf

Energy characterization and optimization of image sensing toward continuous mobile vision

This paper presents a 2.1 M pixel, 120 frame/s CMOS image sensor with column-parallel delta-sigma (ΔΣ) ADC architecture. The use of a second-order ΔΣ ADC improves the conversion speed while reducing the random noise (RN) level as well. The ΔΣ ADC employing an inverter-based ΔΣ modulator and a compact decimation filter is accommodated within a fine pixel pitch of 2.25-μm and improves energy efficiency while providing a high frame-rate of 120 frame/s. A prototype image sensor has been fabricated with a 0.13-μm CMOS process. Measurement results show a RN of 2.4 erms- and a dynamic range of 73 dB. The power consumption of the prototype image sensor is only 180 mW. This work achieves the energy efficiency of 1.7 e-·nJ.

A 2.1 M Pixels, 120 Frame/s CMOS Image Sensor With Column-Parallel $\Delta \Sigma$ ADC Architecture

A 20-bit incremental ADC for battery-powered sensor applications is presented. It is based on an energy-efficient zoom ADC architecture, which employs a coarse 6-bit SAR conversion followed by a fine 15-bit ΔΣ conversion. To further improve its energy efficiency, the ADC employs integrators based on cascoded dynamic inverters for extra gain and PVT tolerance. Dynamic error correction techniques such as auto-zeroing, chopping and dynamic element matching are used to achieve both low offset and high linearity. Measurements show that the ADC achieves 20-bit resolution, 6 ppm INL and 1 μV offset in a conversion time of 40 ms, while drawing only 3.5 μA current from a 1.8 V supply. This corresponds to a state-of-the-art figure-of-merit (FoM) of 182.7 dB. The 0.35 mm2 chip was fabricated in a standard 0.16 μm CMOS process.

/pdf/a-6-3-uw-20-bit-incremental-zoom-adc-with-6-ppm-inl-and-1-uv-2wwwefvtvo.pdf

A 6.3 µW 20 bit Incremental Zoom-ADC with 6 ppm INL and 1 µV Offset

Over the last few years, the demands for high-density and high-speed imaging have increased drastically. Since CMOS image sensors have the advantages of low power consumption and easy system integration, they have become dominant over CCDs in the consumer market [1–4]. A column-parallel ADC architecture is the most widely used ADC in CMOS image sensors for high-speed and low-power operation [2–6]. The column-parallel architecture can be classified as: successive-approximation register (SAR) [2], cyclic [3], single-slope (SS) [4], and delta-sigma (ΔΣ) [5,6] ADCs. Although SAR ADCs have been utilized for high-speed imaging, such as UDTV, they require a DAC in a column, whose area is unacceptably large for consumer electronics with a fine pixel pitch. Cyclic ADCs have also been reported in high-speed imaging, but they have high power consumption and high noise levels. Since SS ADCs provide relatively high resolution with minimum area, they have been widely used in CMOS image sensors. However, SS ADCs require very fast clock signals leading to high power consumption in the case of high-speed imaging. Although ΔΣ ADCs have been investigated for low-noise imaging, they have only been applied for low-speed imaging with large pixel pitch because of the complexity of ΔΣ modulators and following decimation filters.

A 2.1Mpixel 120frame/s CMOS image sensor with column-parallel ΔΣ ADC architecture

This paper proposes a low-power 240 frames/s 2.1 M-pixel CMOS image sensor with column-shared cyclic (CY) ADCs. Two-column shared CY-ADC architecture and two-level stacked ADC placement are employed for low-power and small pixel pitch design. The proposed CY-ADC uses only one OTA and four capacitors. Distributed clocking scheme using cascaded repeaters is proposed to reduce the required peak current. The prototype sensor was fabricated in a 0.13- μm 1P4M process with pixel pitch of 2.25 μm . The designed 10-bit ADC dissipates only 90 μW/channel with 1.5 V supply. The measured DNL and INL are +0.59/-0.83 LSB and +2.8/-3.6 LSB, respectively. The measured maximum pixel rate is 500 Mpixels/s with total power consumption of 300 mW.

A 240-frames/s 2.1-Mpixel CMOS Image Sensor With Column-Shared Cyclic ADCs

The demand for high-quality and high-speed imaging has increased. Column-parallel ≥14b A/D conversion is one of the major approaches to meet these requirements in CMOS image sensors (CIS). Oversampling ADCs such as incremental delta-sigma (I-ΔΣ) ADCs are the solution for a high-resolution ADC having tolerance of analog component errors. Oversampling reduces input temporal noise as well as the quantization error of the ADC itself [1]. However, an I-ΔΣ ADC is also classified as a slow ADC because it requires exponential conversion time to get higher bit resolution. To reduce conversion time, there are two alternative methods: 1) higher-order modulation, and 2) two-step conversion such as an extended-counting technique [2]. In this paper, the extended-counting (EC) method is used since a high-order structure requires more complex hardware and greater power consumption [1,2]. For a general two-step ADC, coarse conversion restricts the total ADC resolution since it determines accuracy of the residue. However, an EC ADC overcomes the accuracy limitation, since the I-ΔΣ can improve its precision through oversampling. Moreover, oversampling also suppresses the noise of the pixel's source follower. Our 14b EC ADC is a blend of a 1st-order I-ΔΣ ADC and a cyclic ADC to simultaneously get high resolution and high speed.

A 14b extended counting ADC implemented in a 24Mpixel APS-C CMOS image sensor

An image sensor includes a pixel array and a plurality of pairs of column lines. The pixel array includes a plurality of unit pixel areas arranged in a plurality of rows and columns. Each of the unit pixel areas includes a readout circuit connected to a corresponding pair of column lines, and first and second photo-electric conversion devices sharing the readout circuit. Each of the unit pixel areas is configured to output a first pixel signal corresponding to a photoelectron generated by the first photo-electric conversion device through the first column line, and to output a second pixel signal corresponding to a photoelectron generated by the second photo-electric conversion device through the second column line.

Kwi-sung Yoo

Papers

A 2.1 M Pixels, 120 Frame/s CMOS Image Sensor With Column-Parallel $\Delta \Sigma$ ADC Architecture

A 2.1Mpixel 120frame/s CMOS image sensor with column-parallel ΔΣ ADC architecture

A 240-frames/s 2.1-Mpixel CMOS Image Sensor With Column-Shared Cyclic ADCs

A 14b extended counting ADC implemented in a 24Mpixel APS-C CMOS image sensor

Unit pixels configured to output different pixel signals through different lines and image sensors including the same