A Color Vision-Based Lane Tracking System for Autonomous Driving on Unmarked Roads
Summary (4 min read)
1.1. Motivation for Autonomous Driving Systems
- The deployment of Autonomous Driving Systems is a challenging topic that has focused the interest of research institutions all across the world since the mid eighties.
- Apart from the obvious advantages related to safety increase, such as accident rate reduction and human life savings, there are other benefits that could clearly derive from automatic driving.
- Thus, on one hand, vehicles keeping a short but reliable safety distance by automatic means allow to increase the capacity of roads and highways.
- Likewise, automatic cooperative driving of vehicle fleets involved in the transportation of heavy loads can lead to notable industrial cost reductions.
1.2. Autonomous Driving on Highways and Extraurban Roads
- The techniques deployed for lane tracking in this kind of scenarios are similar to those developed for road tracking in highways and structured roads, as long as they face common problems.
- The group at the Universitat der Bundeswehr, Munich, headed by E. Dickmanns has also developed a remarkable number of works on this topic since the early 80’s.
- Likewise, another similar system can be found in Lutzeler and Dickmanns (2000) and Gregor et al. (2002), where a real autonomous system for Intelligent Navigation in a network of unmarked roads and intersections is designed and implemented using edge detectors for lane tracking.
- The complete navigation system was implemented on BABIECA, an electric Citroen Berlingo commercial prototype as depicted in Fig.
- Additionally, a live demonstration exhibiting the system capabilities on autonomous driving was also carried out during the IEEE Conference on Intelligent Vehicles 2002, in a private circuit located at Satory , France.
2.1. Region of Interest
- Nonetheless, the use of temporal filtering techniques (as described in the following sections) allows to obtain finer resolution estimations.
- The probability of finding the most relevant road features is assured to be high by making use of a priori knowledge on the road shape, according to the parabolic road model proposed.
- Thus, in most cases the region of interest is reduced to some portion of image surrounding the road edges estimated in the previous iteration of the algorithm.
- This is a valid assumption for road tracking applications heavily relying on the detection of lane markers that represent the road edges.
- This restriction permits to remove nonrelevant elements from the image such as the sky, trees, buildings, etc.
2.2. Road Features
- The combined use of color and shape restrictions provides the essential information required to drive on non structured roads.
- This makes highly recommendable the use of a color space where a clear separation between intensity and color information can be established.
- Hue represents the impression related to the predominant wavelength in the perceived color stimulus.
- This could save some computing time by avoiding going through the trigonometry.
- 1990; Rodriguez et al., 1998), the HSI color space has exhibited superior performance in image segmentation problems as demonstrated in Ikonomakis et al. (2000).
2.3. Road Model
- The use of a road model eases the reconstruction of the road geometry and permits to filter the data computed during the features searching process.
- More concretely, the use of parabolic functions to model the projection of the road edges onto the image plane has been proposed and successfully tested in previous works (Schneiderman and Nashman, 1994).
- A second order polynomial model has only three adjustable coefficients, also known as Simplicity.
- Discontinuities in the road model are only encountered in road intersections and, particularly, in crossroads.
- The adjustable parameters of the several parabolic functions are continuously updated at each iteration of the algorithm using a well known least squares estimator, as will be described later.
2.4. Road Segmentation
- Image segmentation must be carried out by exploiting the cylindrical distribution of color features in the HSI color space, bearing in mind that the separation between road and no road color characteristics is nonlinear.
- According to this, pixels are divided into chromatic and achromatic as proposed in Ikonomakis et al. (2000).
- This turns the segmentation stage into a position dependant process.
- Thus, for pixels clearly located out of the road trajectory, the chromatic and luminance distances to the road pattern color features should be very small in order to effectively be segmented as part of the road.
- Seven a priori road models are utilized for this purpose, as depicted in Fig.
2.5. Handling Shadows and Brightness
- Shadows and brightness on the road are admittedly the greatest difficulty in vision based systems operating in outdoor environments (Bertozzi and Broggi, 1998).
- B ≥ 1 3 I ≤ Iroad,avg − 2 · σroad (15) where b stands for the normalized blue component; Iroad,avg represents the average intensity value of all road pixels, and σroad is the standard deviation of the intensity distribution of road pixels.
- This technique permits to enhance the road segmentation in presence of shadows, and remarkably contributes to improve the robustness of the color adaptation process, particularly in stretches of road largely covered by shadows.
- Analytically the condition is formulated in Eq. (16).
- The improvement achieved by attenuating both brightness and shadows, as described, permits to handle real images in real and complex situations with an extraordinary high performance, becoming an outstanding point of this work.
2.6. Estimation of Road Edges and Width
- The estimation of the skeleton lines of the road and its edges is carried out basing on parabolic functions, as previously described.
- These polynomial functions are the basis to obtain the lateral and orientation error of the vehicle with respect to the center of the lane.
- On the other hand, ŷl(0), ŷc(0), ŷr (0) are the initial estimations for the left edge, right edge, and skeleton lines of the road, respectively, while yli , yri,yci stand for the left edge, right edge, and skeleton lines of basic pattern i .
2.6.2. Estimation of the Skeleton Lines of the Road.
- The skeleton lines of the road at current time instant, ŷc(t), is estimated based on the segmented low resolution image and the previously estimated road trajectory, ŷc(t − 1).
- Thus, the estimation is realized in three steps as described below.
- The estimation of road edges is realized using the same filtering technique described in the previous section.
- For each line in the area of interest, the closest measurements to the middle of the left edge validation area, defined by ŷc(t)−Ŵ (t −1)/2, and right edge validation area, defined by ŷc(t) +.
- An individual road width measure wi is obtained for each line in the region of interest, by computing the difference between the left and right edges (ŷl(t)|x=xi and ŷr (t)|x=xi , respectively) as expressed in Eq. (20).
2.7. Road Color Features Update
- After completing the road edges and width estimation process, the HSI color features of the road pattern are consequently updated so as to account for changes in road appearance and illumination.
- Intuitively, pixels close to the skeleton lines of the road present color features that highly represent the road color pattern.
- Obviously, the selected pixels are only validated if they have been segmented as road pixels at the current iteration.
- The adaptation process described in this section proves to be crucial in practice to keep the segmentation algorithm under stable performance upon illumination changing conditions and color varying asphalt.
- The complete road tracking scheme is graphically summarised in the flow diagram depicted in Fig. 24.
2.8. Discussion of the Method
- The global objective of this section is to put the road tracking algorithm under test in varied real circumstances.
- As appreciated from observation of Fig. 25, the road edges can be neatly distinguished in the segmented image, allowing a clear estimation in real experiments.
- The results obtained in presence of other vehicles parked on the left hand side of the road are illustrated in Fig. 27(b).
- All pixels in the image tend to have similar intensity values, and thus, color differences in the HSI chromatic plane become crucial for segmentation purposes.
- This situation derives in the appearance of dark spots on the road due to wet areas, as depicted in Fig. 31.
3. Implementation and Results
- The complete navigation system described in the previous sections has been implemented on the so-called Babieca prototype vehicle, depicted in Fig. 1, that has been modified to allow for automatic velocity and steering control at a maximum speed of 90 km/h, using the non linear control law developed in Sotelo (2001).
- As stated in Section 1, a live demonstration exhibiting the system capabilities on autonomous driving was also carried out during the IEEE Conference on Intelligent Vehicles 2002, in a private circuit located at Satory , France.
- In order to complete the graphical results depicted in the previous sections, and to illustrate the global behavior of the complete navigation system implemented on Babieca, some general results are shown next.
- During the tests, the reference vehicle velocity is assumed to be kept constant by the velocity controller.
- To complete these results a wide set of video files demonstrating the operational performance of the system in real tests can be retrieved from ftp://www. depeca.uah.es/pub/vision.
4. Conclusions
- The road segmentation algorithm based on the HSI color space and 2D-spatial constraints, as described in this work, has successfully proved to provide correct estimations for the edges and width of non-structured roads, i.e., roads without lane markers.
- The practical results discussed above also support the validity of the method for different environmental and weather conditions, as demonstrated so far.
- Otherwise, there will be long shadows but the system performs well.
- The most remarkable feature of the road tracking scheme described in this work is its ability to correctly deal with non-structured roads by performing a non-supervised color-based road segmentation process.
- Nonetheless, a lot of work still remains to be done until a completely robust and reliable autonomous system can be fully deployed in real conditions.
Did you find this useful? Give us your feedback
Citations
7 citations
Cites methods from "A Color Vision-Based Lane Tracking ..."
...The first algorithm is the HSI road detection (RD) algorithm proposed in [ 25 ] and used in [21]....
[...]
6 citations
6 citations
6 citations
Cites background from "A Color Vision-Based Lane Tracking ..."
...Some vision approaches are based on low-level features [2][3][4]....
[...]
6 citations
References
1,088 citations
"A Color Vision-Based Lane Tracking ..." refers background or methods in this paper
...Shadows and brightness on the road are admittedly the greatest difficulty in vision based systems operating in outdoor environments (Bertozzi and Broggi, 1998)....
[...]
...…during the last decade by the research groups at the UBM (Dickmanns et al., 1994; Lutzeler and Dickmanns, 1998) and Daimler-Benz (Franke et al., 1998), or the GOLD system (Bertozzi and Broggi, 1998; Broggi et al., 1999) implemented on the ARGO autonomous vehicle at the Universita di Parma....
[...]
...Likewise, automatic cooperative driving of vehicle fleets involved in the transportation of heavy loads can lead to notable industrial cost reductions....
[...]
780 citations
"A Color Vision-Based Lane Tracking ..." refers methods in this paper
...Although the RGB color space has been successfully used in previous works dealing with road segmentation ( Thorpe, 1990; Rodriguez et al., 1998), the HSI color space has exhibited superior performance in image segmentation problems as demonstrated in Ikonomakis et al. (2000)....
[...]
...On one hand, the RGB color space has been extensively tested and used in previous road tracking applications on non-structured roads ( Thorpe, 1990; Crisman and Thorpe, 1991; Rodriguez et al., 1998)....
[...]
...Among them are the SCARF and UNSCARF systems ( Thorpe, 1990 ) designed to extract the road shape basing on the study of homogeneous regions from a color image....
[...]
648 citations
"A Color Vision-Based Lane Tracking ..." refers background in this paper
...Thus, autonomous guidance of vehicles on either marked or unmarked roads demonstrated its first results in Dickmanns and Zapp (1986) and Dickmanns and Mysliwetz (1992) where nine road and vehicle parameters were recursively estimated following the 4D approach on 3D scenes....
[...]
508 citations
"A Color Vision-Based Lane Tracking ..." refers background in this paper
...On the other hand, the normalized blue component is generally predominant over the normalized red and green components, as discussed in Pomerleau (1993)....
[...]
...Likewise, automatic cooperative driving of vehicle fleets involved in the transportation of heavy loads can lead to notable industrial cost reductions....
[...]
...The ALVINN (Autonomous Land Vehicle In a Neural Net) (Pomerleau, 1993) system is also able to follow unmarked roads after a proper training phase on the particular roads where the vehicle must navigate....
[...]
448 citations
"A Color Vision-Based Lane Tracking ..." refers background in this paper
...As discussed in Bertozzi et al. (2000) due to the existence of physical and continuity constraints derived from vehicle motion and road design, the analysis of the whole image can be replaced by the analysis of a specific portion of it, namely the region of interest....
[...]
...In order to deal with this problem, some authors propose to improve the dynamic range of visual cameras (Bertozzi et al., 2000) so as to tackle strong luminance changes, when entering or exiting tunnels for instance, or to enhance the sensitiveness of cameras to the blue component of colors....
[...]