The Kalman filter approach to recursive state estimation making use of dynamic models for the motion of massive objects has been extended to image sequence processing. This confines image processing to the last frame of the sequence only, and derives a direct spatial interpretation including spatial velocity components by smoothing integrations of prediction errors. Results are presented for road-vehicle guidance at high speeds including obstacle detection and monocular relative spatial state estimation. The corresponding data-processing architecture is discussed; the system has been implemented on a MIMD parallel processing system. Speeds up to 100 km/h have been demonstrated. >

An integrated spatio-temporal approach to automatic visual guidance of autonomous vehicles

Introduction Basic Relations: Image Sequences - 'the World' Subjects and Subject Classes Application Domains, Missions and Situations Extraction of Visual Features Recursive State Estimation Beginnings of Spatio-temporal Road and Ego-state Recognition The Initialization Challenge in Dynamic Scene Understanding Recursive Estimation of Road Parameters and Ego-state while Cruising Perception of Crossroads Perception of Obstacles and Other Vehicles Sensor Requirements for Flexible Perception of Road Scenes Integrated Knowledge Representations for Dynamic Vision Mission Performance, Experimental Results Conclusions and Outlook

Dynamic Vision for Perception and Control of Motion

YARF, like several other road following systems, models the road as features which form concentric circular arcs lying in a flat ground plane. Two approximations are made in order to use a linear estimation technique to determine the spine parameters. The paper demonstrates that errors due to these approximations are reduced substantially by estimating the spine parameters in a data-dependent coordinate system rather than in a vehicle-centered coordinate system. It discusses the problem of estimating the spine parameters in the presence of outlying data. Using standard least squares estimation can result in the vehicle drifting off the road due to errors in the parameter estimates caused by such contaminants in the data. The use of least median squares estimation to overcome these problems is discussed. >

Representation and recovery of road geometry in YARF

An efficient distributed processing scheme has been developed for visual road boundary tracking by 'VaMoRs', a testbed vehicle for autonomous mobility and computer vision Ongoing work described here is directed to improving the robustness of the road boundary detection process in the presence of shadows, ill-defined edges and other disturbing real world effects The system structure and the techniques applied for real-time scene analysis are presented along with experimental results All subfunctions of road boundary detection for vehicle guidance, such as edge extraction, feature aggregation and camera pointing control, are executed in parallel by an onboard multiprocessor system On the image processing level local oriented edge extraction is performed in multiple 'windows', tighly controlled from a hierarchically higher, modelbased level The interpretation process involving a geometric road model and the observer's relative position to the road boundaries is capable of coping with ambiguity in measurement data By using only selected measurements to update the model parameters even high noise levels can be dealt with and misleading edges be rejected

Distributed Scene Analysis For Autonomous Road Vehicle Guidance

Efficient real-time visual perception in civilized natural environments (e.g. road networks) has to take advantage of foveal – peripheral differentiation for data economy and of active gaze control for a number of benefits. 1. Inertial gaze stabilization considerably alleviates the evaluation of image sequences taken by cameras with stronger tele-lenses; it allows a reduction in angular disturbances from rough ground by at least an order of magnitude with simple negative angular rate data feedback. 2. Visual tracking of fast moving objects reduces motion blur for these objects. – 3. In the near range, a large field of view is mandatory, however, only coarse angular resolution is sufficient; with a field of view (f.o.v.) > ~ 100°, both the region in front of and to the side of the vehicle may be viewed simultaneously. For own behavior decision, motion behaviors of objects both in the wide f.o.v. nearby and in several regions of special interest further away have to be understood in conjunction. In order to achieve this efficiently, three distinct visual processes with specific knowledge bases have to be employed in a consecutive way. In the wide f.o.v., bottom-up feature extraction has to answer the question: ‘Is there anything of special interest?’ The corresponding feature extraction operators are domain-specific. On initialization, they have to give indications of objects of interest all over the image. Stable feature aggregations over several cycles have to trigger object hypotheses for the second stage; these regions may then be discarded for stage 1. Stage 2 works on single objects, however, on multiple of these in parallel. When looking almost parallel to the ground this is necessary for proper scaling, since each line in the image represents a different distance on the ground plane. For this reason, feature extractors and state estimators for each object have to be tuned specifically. Representing 3-D objects in 3-D space and time allows exploiting the first order derivative matrix of the perspective mapping process (the so-called ‘Jacobian’) and spatial interpretation despite the fact that range has been lost in each single image point. The parallel processes of stage 2 yield best estimates of the relative state ‘here and now’ for all objects observed. They are symbolically represented in a scene tree known from computer graphics. Stage 3 works on time series of these symbolic data in order to more deeply understand motion sequences on larger spatial and temporal scales (maneuvers, mission elements) affecting own decision making. For more autonomous, flexible use, both perceptual and behavioral capabilities are represented explicitly. Experimental results will be given for the test vehicles VaMoRs (a 5-ton van) and VaMP (Mercedes 500 SEL).

Three-Stage Visual Perception for Vertebrate-type Dynamic Machine Vision

This paper describes a real-time road following and road junction detection vision system for autonomous vehicles. Vision-guided road following requires extracting road boundaries from images in real-time to guide the navigation of autonomous vehicles on the roadway. We use a histogram-based pixel classification algorithm to classify road and non-road regions in the image. The most likely road region is selected and a polygonal representation of the detected road region boundary is used as the input to a geometric reasoning module that performs model-based reasoning to accurately identify consistent road segments and road junctions. In this module, local geometric supports for each road edge segment are collected and recorded and a global consistency checking is performed' to obtain a consistent interpretation of the raw data. Limited cases of incorrect image segmentation due to shadows or unusual road conditions can be detected and corrected based on the road model. Similarly, road junctions can be detected using the same principle. The real-time road following vision system has been implemented on a high-speed image processor connected to a host computer. We have tested our road following vision system and vehicle control system on a gravel road.

Gary Phipps

Papers

A Real-Time Road Following Vision System For Autonomous Vehicles