A Color Vision-Based Lane Tracking System for Autonomous Driving on Unmarked Roads
Summary (4 min read)
1.1. Motivation for Autonomous Driving Systems
- The deployment of Autonomous Driving Systems is a challenging topic that has focused the interest of research institutions all across the world since the mid eighties.
- Apart from the obvious advantages related to safety increase, such as accident rate reduction and human life savings, there are other benefits that could clearly derive from automatic driving.
- Thus, on one hand, vehicles keeping a short but reliable safety distance by automatic means allow to increase the capacity of roads and highways.
- Likewise, automatic cooperative driving of vehicle fleets involved in the transportation of heavy loads can lead to notable industrial cost reductions.
1.2. Autonomous Driving on Highways and Extraurban Roads
- The techniques deployed for lane tracking in this kind of scenarios are similar to those developed for road tracking in highways and structured roads, as long as they face common problems.
- The group at the Universitat der Bundeswehr, Munich, headed by E. Dickmanns has also developed a remarkable number of works on this topic since the early 80’s.
- Likewise, another similar system can be found in Lutzeler and Dickmanns (2000) and Gregor et al. (2002), where a real autonomous system for Intelligent Navigation in a network of unmarked roads and intersections is designed and implemented using edge detectors for lane tracking.
- The complete navigation system was implemented on BABIECA, an electric Citroen Berlingo commercial prototype as depicted in Fig.
- Additionally, a live demonstration exhibiting the system capabilities on autonomous driving was also carried out during the IEEE Conference on Intelligent Vehicles 2002, in a private circuit located at Satory , France.
2.1. Region of Interest
- Nonetheless, the use of temporal filtering techniques (as described in the following sections) allows to obtain finer resolution estimations.
- The probability of finding the most relevant road features is assured to be high by making use of a priori knowledge on the road shape, according to the parabolic road model proposed.
- Thus, in most cases the region of interest is reduced to some portion of image surrounding the road edges estimated in the previous iteration of the algorithm.
- This is a valid assumption for road tracking applications heavily relying on the detection of lane markers that represent the road edges.
- This restriction permits to remove nonrelevant elements from the image such as the sky, trees, buildings, etc.
2.2. Road Features
- The combined use of color and shape restrictions provides the essential information required to drive on non structured roads.
- This makes highly recommendable the use of a color space where a clear separation between intensity and color information can be established.
- Hue represents the impression related to the predominant wavelength in the perceived color stimulus.
- This could save some computing time by avoiding going through the trigonometry.
- 1990; Rodriguez et al., 1998), the HSI color space has exhibited superior performance in image segmentation problems as demonstrated in Ikonomakis et al. (2000).
2.3. Road Model
- The use of a road model eases the reconstruction of the road geometry and permits to filter the data computed during the features searching process.
- More concretely, the use of parabolic functions to model the projection of the road edges onto the image plane has been proposed and successfully tested in previous works (Schneiderman and Nashman, 1994).
- A second order polynomial model has only three adjustable coefficients, also known as Simplicity.
- Discontinuities in the road model are only encountered in road intersections and, particularly, in crossroads.
- The adjustable parameters of the several parabolic functions are continuously updated at each iteration of the algorithm using a well known least squares estimator, as will be described later.
2.4. Road Segmentation
- Image segmentation must be carried out by exploiting the cylindrical distribution of color features in the HSI color space, bearing in mind that the separation between road and no road color characteristics is nonlinear.
- According to this, pixels are divided into chromatic and achromatic as proposed in Ikonomakis et al. (2000).
- This turns the segmentation stage into a position dependant process.
- Thus, for pixels clearly located out of the road trajectory, the chromatic and luminance distances to the road pattern color features should be very small in order to effectively be segmented as part of the road.
- Seven a priori road models are utilized for this purpose, as depicted in Fig.
2.5. Handling Shadows and Brightness
- Shadows and brightness on the road are admittedly the greatest difficulty in vision based systems operating in outdoor environments (Bertozzi and Broggi, 1998).
- B ≥ 1 3 I ≤ Iroad,avg − 2 · σroad (15) where b stands for the normalized blue component; Iroad,avg represents the average intensity value of all road pixels, and σroad is the standard deviation of the intensity distribution of road pixels.
- This technique permits to enhance the road segmentation in presence of shadows, and remarkably contributes to improve the robustness of the color adaptation process, particularly in stretches of road largely covered by shadows.
- Analytically the condition is formulated in Eq. (16).
- The improvement achieved by attenuating both brightness and shadows, as described, permits to handle real images in real and complex situations with an extraordinary high performance, becoming an outstanding point of this work.
2.6. Estimation of Road Edges and Width
- The estimation of the skeleton lines of the road and its edges is carried out basing on parabolic functions, as previously described.
- These polynomial functions are the basis to obtain the lateral and orientation error of the vehicle with respect to the center of the lane.
- On the other hand, ŷl(0), ŷc(0), ŷr (0) are the initial estimations for the left edge, right edge, and skeleton lines of the road, respectively, while yli , yri,yci stand for the left edge, right edge, and skeleton lines of basic pattern i .
2.6.2. Estimation of the Skeleton Lines of the Road.
- The skeleton lines of the road at current time instant, ŷc(t), is estimated based on the segmented low resolution image and the previously estimated road trajectory, ŷc(t − 1).
- Thus, the estimation is realized in three steps as described below.
- The estimation of road edges is realized using the same filtering technique described in the previous section.
- For each line in the area of interest, the closest measurements to the middle of the left edge validation area, defined by ŷc(t)−Ŵ (t −1)/2, and right edge validation area, defined by ŷc(t) +.
- An individual road width measure wi is obtained for each line in the region of interest, by computing the difference between the left and right edges (ŷl(t)|x=xi and ŷr (t)|x=xi , respectively) as expressed in Eq. (20).
2.7. Road Color Features Update
- After completing the road edges and width estimation process, the HSI color features of the road pattern are consequently updated so as to account for changes in road appearance and illumination.
- Intuitively, pixels close to the skeleton lines of the road present color features that highly represent the road color pattern.
- Obviously, the selected pixels are only validated if they have been segmented as road pixels at the current iteration.
- The adaptation process described in this section proves to be crucial in practice to keep the segmentation algorithm under stable performance upon illumination changing conditions and color varying asphalt.
- The complete road tracking scheme is graphically summarised in the flow diagram depicted in Fig. 24.
2.8. Discussion of the Method
- The global objective of this section is to put the road tracking algorithm under test in varied real circumstances.
- As appreciated from observation of Fig. 25, the road edges can be neatly distinguished in the segmented image, allowing a clear estimation in real experiments.
- The results obtained in presence of other vehicles parked on the left hand side of the road are illustrated in Fig. 27(b).
- All pixels in the image tend to have similar intensity values, and thus, color differences in the HSI chromatic plane become crucial for segmentation purposes.
- This situation derives in the appearance of dark spots on the road due to wet areas, as depicted in Fig. 31.
3. Implementation and Results
- The complete navigation system described in the previous sections has been implemented on the so-called Babieca prototype vehicle, depicted in Fig. 1, that has been modified to allow for automatic velocity and steering control at a maximum speed of 90 km/h, using the non linear control law developed in Sotelo (2001).
- As stated in Section 1, a live demonstration exhibiting the system capabilities on autonomous driving was also carried out during the IEEE Conference on Intelligent Vehicles 2002, in a private circuit located at Satory , France.
- In order to complete the graphical results depicted in the previous sections, and to illustrate the global behavior of the complete navigation system implemented on Babieca, some general results are shown next.
- During the tests, the reference vehicle velocity is assumed to be kept constant by the velocity controller.
- To complete these results a wide set of video files demonstrating the operational performance of the system in real tests can be retrieved from ftp://www. depeca.uah.es/pub/vision.
4. Conclusions
- The road segmentation algorithm based on the HSI color space and 2D-spatial constraints, as described in this work, has successfully proved to provide correct estimations for the edges and width of non-structured roads, i.e., roads without lane markers.
- The practical results discussed above also support the validity of the method for different environmental and weather conditions, as demonstrated so far.
- Otherwise, there will be long shadows but the system performs well.
- The most remarkable feature of the road tracking scheme described in this work is its ability to correctly deal with non-structured roads by performing a non-supervised color-based road segmentation process.
- Nonetheless, a lot of work still remains to be done until a completely robust and reliable autonomous system can be fully deployed in real conditions.
Did you find this useful? Give us your feedback
Citations
327 citations
Cites background or methods from "A Color Vision-Based Lane Tracking ..."
...Thus, additional constraints must be included to improve their performance, e.g., in [ 13 ], Sotelo et al. use road shape restrictions and ad hoc postprocessing to recover undetected shadowed areas....
[...]
...For comparison, the image sequence SRD is processed using an HSI-based road-detection algorithm inspired in [7] and [ 13 ]....
[...]
...Other works in the field [6], [ 13 ] follow similar assumptions....
[...]
194 citations
184 citations
109 citations
Cites background or methods from "A Color Vision-Based Lane Tracking ..."
...The last two algorithms are photometric–based approaches: the illuminant–invariant algorithm in [18] and the HSI based algorithm proposed in [1] and used in [5]....
[...]
...The two most popular color spaces, that have proved to be robust to minor illuminant changes, are HSV [1, 5] and normalized RGB [2]....
[...]
...In general, vision–based methods use low-level features for road detection [1, 2, 3, 4]....
[...]
...Algorithms based on transformed color spaces (i.e., HSV [5, 1] or rg [2]) are, to a certain degree still sensitive to lighting variations such as strong shadows and highlights....
[...]
..., HSV [5, 1] or rg [2]) are, to a certain degree still sensitive to lighting variations such as strong shadows and highlights....
[...]
93 citations
References
445 citations
"A Color Vision-Based Lane Tracking ..." refers background or methods in this paper
...On one hand, the RGB color space has been extensively tested and used in previous road tracking applications on non-structured roads (Thorpe, 1990; Crisman and Thorpe, 1991; Rodriguez et al., 1998)....
[...]
...Although the RGB color space has been successfully used in previous works dealing with road segmentation (Thorpe, 1990; Rodriguez et al., 1998), the HSI color space has exhibited superior performance in image segmentation problems as demonstrated in Ikonomakis et al. (2000)....
[...]
...Likewise, automatic cooperative driving of vehicle fleets involved in the transportation of heavy loads can lead to notable industrial cost reductions....
[...]
...Among them are the SCARF and UNSCARF systems (Thorpe, 1990) designed to extract the road shape basing on the study of homogeneous regions from a color image....
[...]
432 citations
"A Color Vision-Based Lane Tracking ..." refers background or methods in this paper
...Likewise, automatic cooperative driving of vehicle fleets involved in the transportation of heavy loads can lead to notable industrial cost reductions....
[...]
...That’s the case of some well known and prestigious systems such as RALPH (Pomerleau and Jockem, 1996) (Rapid Adapting Lateral Position Handler), developed on the Navlab vehicle at the Robotics Institute of the Carnegie Mellon University, the impressive unmanned vehicles developed during the last decade by the research groups at the UBM (Dickmanns et al., 1994; Lutzeler and Dickmanns, 1998) and Daimler-Benz (Franke et al., 1998), or the GOLD system (Bertozzi and Broggi, 1998; Broggi et al., 1999) implemented on the ARGO autonomous vehicle at the Universita di Parma....
[...]
...…vehicles developed during the last decade by the research groups at the UBM (Dickmanns et al., 1994; Lutzeler and Dickmanns, 1998) and Daimler-Benz (Franke et al., 1998), or the GOLD system (Bertozzi and Broggi, 1998; Broggi et al., 1999) implemented on the ARGO autonomous vehicle at the…...
[...]
332 citations
"A Color Vision-Based Lane Tracking ..." refers background in this paper
...Thus, autonomous guidance of vehicles on either marked or unmarked roads demonstrated its first results in Dickmanns and Zapp (1986) and Dickmanns and Mysliwetz (1992) where nine road and vehicle parameters were recursively estimated following the 4D approach on 3D scenes....
[...]
324 citations
"A Color Vision-Based Lane Tracking ..." refers background in this paper
...The last statement can be regarded as the slow varying road width assumption, widely used in previous works on road tracking (Kim et al., 1995; Pomerleau and Jockem, 1996)....
[...]
...On the other hand, a remarkable saving in fuel expenses can be achieved by automatically controlling vehicles velocity so as to keep a soft acceleration profile....
[...]
...That’s the case of some well known and prestigious systems such as RALPH (Pomerleau and Jockem, 1996) (Rapid Adapting Lateral Position Handler), developed on the Navlab vehicle at the Robotics Institute of the Carnegie Mellon University, the impressive unmanned vehicles developed during the last decade by the research groups at the UBM (Dickmanns et al., 1994; Lutzeler and Dickmanns, 1998) and Daimler-Benz (Franke et al., 1998), or the GOLD system (Bertozzi and Broggi, 1998; Broggi et al., 1999) implemented on the ARGO autonomous vehicle at the Universita di Parma....
[...]
...That’s the case of some well known and prestigious systems such as RALPH (Pomerleau and Jockem, 1996) (Rapid Adapting Lateral Position Handler), developed on the Navlab vehicle at the Robotics Institute of the Carnegie Mellon University, the impressive unmanned vehicles developed during the last…...
[...]
244 citations
"A Color Vision-Based Lane Tracking ..." refers background or methods in this paper
...…Institute of the Carnegie Mellon University, the impressive unmanned vehicles developed during the last decade by the research groups at the UBM (Dickmanns et al., 1994; Lutzeler and Dickmanns, 1998) and Daimler-Benz (Franke et al., 1998), or the GOLD system (Bertozzi and Broggi, 1998; Broggi…...
[...]
...That’s the case of some well known and prestigious systems such as RALPH (Pomerleau and Jockem, 1996) (Rapid Adapting Lateral Position Handler), developed on the Navlab vehicle at the Robotics Institute of the Carnegie Mellon University, the impressive unmanned vehicles developed during the last decade by the research groups at the UBM (Dickmanns et al., 1994; Lutzeler and Dickmanns, 1998) and Daimler-Benz (Franke et al....
[...]
...Among the different possibilities found in the literature, models relaying on clothoids (Dickmanns et al., 1994) and polynomial expressions have extensively exhibited high performance in the field of road tracking....
[...]
...That’s the case of some well known and prestigious systems such as RALPH (Pomerleau and Jockem, 1996) (Rapid Adapting Lateral Position Handler), developed on the Navlab vehicle at the Robotics Institute of the Carnegie Mellon University, the impressive unmanned vehicles developed during the last decade by the research groups at the UBM (Dickmanns et al., 1994; Lutzeler and Dickmanns, 1998) and Daimler-Benz (Franke et al., 1998), or the GOLD system (Bertozzi and Broggi, 1998; Broggi et al., 1999) implemented on the ARGO autonomous vehicle at the Universita di Parma....
[...]