The concept of a real-time range camera without moving parts is described, based on the time-of-flight (TOF) principle. It operates with modulated visible and near-infrared radiation, which is detected and demodulated simultaneously by a 2-D array of lock-in pixels employing the charge-coupled device principle. Each pixel individually measures the amplitude, offset and phase of the received radiation. The theoretical resolution limit of this TOF range camera is derived, which depends on the square root of the detected background radiation and the inverse of the modulation amplitude. Actual measurements of 3-D sequences acquired at 10 range images per second show excellent agreement between our theory and the observed results. A range resolution of a few centimeters over a range of 10 m, with an illumination power of a few hundreds of milliwatts is obtained in laboratory scenes for noncooperative, diffusely reflecting objects.

Solid-state time-of-flight range camera

This paper reviews the state-of-the art in the field of lock-in time-of-flight (ToF) cameras, their advantages, their limitations, the existing calibration methods, and the way they are being used, sometimes in combination with other sensors. Even though lock-in ToF cameras provide neither higher resolution nor larger ambiguity-free range compared to other range map estimation systems, advantages such as registered depth and intensity data at a high frame rate, compact design, low weight, and reduced power consumption have motivated their increasing usage in several research areas, such as computer graphics, machine vision, and robotics.

/pdf/lock-in-time-of-flight-tof-cameras-a-survey-365c0dnen2.pdf

Lock-in Time-of-Flight (ToF) Cameras: A Survey

A new miniaturized camera system that is capable of 3-dimensional imaging in real-time is presented. The compact imaging device is able to entirely capture its environment in all three spatial dimensions. It reliably and simultaneously delivers intensity data as well as range information on the objects and persons in the scene. The depth measurement is based on the time-of-flight (TOF) principle. A custom solid-state image sensor allows the parallel measurement of the
phase, offset and amplitude of a radio frequency (RF) modulated light field that is emitted by the system and reflected back by the camera surroundings without requiring any mechanical scanning parts. In this paper, the theoretical background of the implemented TOF principle is presented, together with the technological requirements and detailed
practical implementation issues of such a distance measuring system. Furthermore, the schematic overview of the complete 3D-camera system is provided. The experimental test results are presented and discussed. The present camera system can achieve sub-centimeter depth resolution for a wide range of operating conditions. A miniaturized version of such a 3D-solid-state camera, the SwissRanger 2, is presented as an example, illustrating the possibility of
manufacturing compact, robust and cost effective ranging camera products for 3D imaging in real-time.

An all-solid-state optical range camera for 3D real-time imaging with sub-centimeter depth resolution (SwissRanger)

Die vorliegende Erfindung betrifft ein Verfahren zur Bestimmung der Phasen- und/oder Amplitudeninformation einer elektromagnetischen Welle und eine entsprechende Vorrichtung. Um eine raumliche Tiefenauflosung der mit einem solchen Verfahren bei einer Vorrichtung gewonnenen Bildinformation zu erhalten, wird erfindungsgemas ein Verfahren vorgeschlagen, bei dem eine elektromagnetische Welle auf die Oberflache eines mindestens ein Pixel aufweisenden photonischen Mischelementes eingestrahlt wird, wobei das Pixel mindestens zwei lichtempfindliche Modulationsphotogates Gam und Gbm und zugeordnete Akkumulationsgates Ga und Gb aufweist; bei dem an die Modulationsphotogates Gam und Gbm Modulationsphotogatespannungen Uam(t) und Ubm(t) angelegt werden, die als Uam(t) = Uo + Um(t) und Ubm(t) = Um(t) ausgestaltet sind; bei dem an die Akkumulationsgates Ga und Gb eine Gleichspannung angelegt wird, deren Betrag mindestens so gros wie der Betrag der Summe aus Uo und der Amplitude der Modulationsspannung Um(t) ist; bei dem die in der Raumladungszone der Modulationsphotogates Gam und Gbm von der einfallenden elektromagnetischen Welle erzeugten Ladungstrager in Abhangigkeit von der Polaritat der Modulationsphotogatespannungen Uam(t) und Ubm(t) dem Potentialgefalle eines Driftfeldes ausgesetzt werden und zum entsprechenden Akkumulationsgate Ga oder Gb driften; und bei dem die jeweils zu den Akkumulationsgates Ga und Gb gedrifteten Ladungen qa und qb abgeleitet werden. Das entsprechende photonische Mischelement hat mindestens ein Pixel, welches mindestens zwei lichtempfindliche Modulationsphotogates (Gam, Gbm) und den Modulationsphotogates zugeordnete, gegenuber der einfallenden elektromagnetischen Welle abgeschottete Akkumulationsgates (Ga, Gb) aufweist. Mehrere photonische Mischelemente konnen zu einem Array zusammengeschaltet werden.

Verfahren und vorrichtung zur bestimmung der phasen- und/oder amplitudeninformation einer elektromagnetischen welle

This paper presents a new type of CMOS time-of-flight (TOF) range image sensor using single-layer gates on field oxide structure for photo conversion and charge transfer. This simple structure allows the realization of a dense TOF range imaging array with 1515 mum2 pixels in a standard CMOS process. Only an additional process step to create an n-type buried layer which is necessary for high-speed charge transfer is added to the fabrication process. The sensor operates based on time-delay dependent modulation of photocharge induced by back reflected infrared light pulses from an active illumination light source. To reduce the influence of background light, a small duty cycle light pulse is used and charge draining structures are included in the pixel. The TOF sensor chip fabricated measures a range resolution of 2.35 cm at 30 frames per second and an improvement to 0.74 cm at three frames per second with a pulsewidth of 100 ns.

A CMOS Time-of-Flight Range Image Sensor With Gates-on-Field-Oxide Structure

A new type of CCD image sensor for two-dimensional synchronous detection ("lock-in imager") is presented. The measurement principle allows each pixel to measure the size of the amplitude modulation, the relative phase and the mean brightness level (background) of an oscillating optical wave field. Design, operation and measurement results are presented. A typical application for this sensor would be in 3-D imaging (range measurement) using heterodyne interferometry. >

The lock-in CCD-two-dimensional synchronous detection of light

A fully parallel focal plane charge-coupled device (CCD) array which performs image acquisition and convolution with arbitrary kernels is presented. The real-time programmable spatial convolution is generated for all pixels in parallel during the exposure. The 2-D convolution is performed by shifting a charge pattern in two dimensions and the exposure time is varied in proportion to the weight of each kernel coefficient. The problem of negative weights can be solved by taking the difference of two convolutions each with only positive weights. The CCD was fabricated using a standard CMOS/CCD process. Convolutions have been performed with a variety of linear filters that are commonly used in machine vision. Typical rms deviations from the ideal filter characteristics are between 1-2% of the largest kernel tap value. Results and practical applications of this work are discussed.

A smart CCD image sensor with real-time programmable parallel convolution capabilities

The combination of low-level image processing algorithms with CCD/CMOS based VLSI electronics makes possible the realization of a smart image sensor, which is capable of carrying out 2D feature detection on the image plane The device presented is a low-cost, fast and small solution for real-time, low-level image processing problems

F. Heitger

Papers

The lock-in CCD-two-dimensional synchronous detection of light

A smart CCD image sensor with real-time programmable parallel convolution capabilities

Real-time 2D feature detection with low-level image processing algorithms on smart CCD/CMOS image sensors