The development of an embedded system for real-time correction of fish-eye effect is presented. The fish-eye lens is applied to driver assistance video systems because of its wide-angled view. A large field of view can reduce the number of cameras needed for video system and their cost, installation, maintenance and wiring issues. On the other hand, this lens causes inherent radial distortion to image that has to be corrected in real-time with a low-cost and low-power processing platform. This paper proposes a solution that can be easily adapted to different types of lens and camera, and meets real-time constraints with a power budget within 100 mW and a board size of few cm2. Starting from mathematical equations, given by the geometrical optics, a state-of-the-art correction method is presented, then optimizations are introduced at different levels: algorithmic level, where a real-time correction parameter calculation avoids extra non-volatile off-chip memory cards; data transfer level, where a new pixel pair management reduces memory access and storage burden; HW-SW implementation level, where a low-power board has been developed and tested in real automotive scenarios. Other applications of the developed system, such as multi-camera and multi-dimensional video systems, are finally presented.

Low-power DSP system for real-time correction of fish-eye cameras in automotive driver assistance applications

Cameras are excellent ways of non-invasively monitoring the interior and exterior of vehicles. In particular, high speed stereovision and multivision systems are important for transport applications such as driver eye tracking or collision avoidance. This paper addresses the synchronisation problem which arises when multivision camera systems are used to capture the high speed motion common in such applications. An experimental, high-speed tri-vision camera system intended for real-time driver eye-blink and saccade measurement was designed, developed, implemented and tested using prototype, ultra-high dynamic range, automotive-grade image sensors specifically developed by E2V (formerly Atmel) Grenoble SA as part of the European FP6 project – sensation (advanced sensor development for attention stress, vigilance and sleep/wakefulness monitoring). The developed system can sustain frame rates of 59.8 Hz at the full stereovision resolution of 1280 × 480 but this can reach 750 Hz when a 10 k pixel Region of Interest (ROI) is used, with a maximum global shutter speed of 1/48000 s and a shutter efficiency of 99.7%. The data can be reliably transmitted uncompressed over standard copper Camera-Link® cables over 5 metres. The synchronisation error between the left and right stereo images is less than 100 ps and this has been verified both electrically and optically. Synchronisation is automatically established at boot-up and maintained during resolution changes. A third camera in the set can be configured independently. The dynamic range of the 10bit sensors exceeds 123 dB with a spectral sensitivity extending well into the infra-red range. The system was subjected to a comprehensive testing protocol, which confirms that the salient requirements for the driver monitoring application are adequately met and in some respects, exceeded. The synchronisation technique presented may also benefit several other automotive stereovision applications including near and far-field obstacle detection and collision avoidance, road condition monitoring and others.

/pdf/a-high-speed-tri-vision-system-for-automotive-applications-29xyb8wg2c.pdf

A high speed tri-vision system for automotive applications

A real-time system is proposed to acquire from an automotive fish-eye CMOS camera the traffic signs, and provide
their automatic recognition on the vehicle network. Differently from the state-of-the-art, in this work color-detection is
addressed exploiting the HSI color space which is robust to lighting changes. Hence the first stage of the processing
system implements fish-eye correction and RGB to HSI transformation. After color-based detection a noise deletion
step is implemented and then, for the classification, a template-based correlation method is adopted to identify potential
traffic signs, of different shapes, from acquired images. Starting from a segmented-image a matching with templates of
the searched signs is carried out using a distance transform. These templates are organized hierarchically to reduce the
number of operations and hence easing real-time processing for several types of traffic signs. Finally, for the
recognition of the specific traffic sign, a technique based on extraction of signs characteristics and thresholding is
adopted. Implemented on DSP platform the system recognizes traffic signs in less than 150 ms at a distance of about 15
meters from 640x480-pixel acquired images. Tests carried out with hundreds of images show a detection and recognition
rate of about 93%.

Real-time color/shape-based traffic signs acquisition and recognition system

With the increasing demand on driver information and multimedia content, displays and video signalling are getting more and more attention in the automotive industry. The requirement of transmitting video signals includes applications like infotainment displays, dashboard and head-up displays, but also driver assistance systems requiring real time video streams. The Automotive Pixel Link (APIX) Technology has been specifically designed to address the different requirements for video transmission in automotive applications. Optimized for low EMI, the APIX technology offers the ability to combine high speed real-time video data, full-duplex sideband channels and the power supply over a single cable. Especially on multicamera applications, the video information needs to be provided with minimum delay, with special requirements on synchronization and image processing. This article provides an introduction on the APIX technology, followed by the discussion of various requirements of implementing camera based driver assistance systems in the automotive environment.

Thorsten Heimann

Papers

Real-Time Camera Link for Driver Assistance Applications