A Color Vision-Based Lane Tracking System for Autonomous Driving on Unmarked Roads
Summary (4 min read)
1.1. Motivation for Autonomous Driving Systems
- The deployment of Autonomous Driving Systems is a challenging topic that has focused the interest of research institutions all across the world since the mid eighties.
- Apart from the obvious advantages related to safety increase, such as accident rate reduction and human life savings, there are other benefits that could clearly derive from automatic driving.
- Thus, on one hand, vehicles keeping a short but reliable safety distance by automatic means allow to increase the capacity of roads and highways.
- Likewise, automatic cooperative driving of vehicle fleets involved in the transportation of heavy loads can lead to notable industrial cost reductions.
1.2. Autonomous Driving on Highways and Extraurban Roads
- The techniques deployed for lane tracking in this kind of scenarios are similar to those developed for road tracking in highways and structured roads, as long as they face common problems.
- The group at the Universitat der Bundeswehr, Munich, headed by E. Dickmanns has also developed a remarkable number of works on this topic since the early 80’s.
- Likewise, another similar system can be found in Lutzeler and Dickmanns (2000) and Gregor et al. (2002), where a real autonomous system for Intelligent Navigation in a network of unmarked roads and intersections is designed and implemented using edge detectors for lane tracking.
- The complete navigation system was implemented on BABIECA, an electric Citroen Berlingo commercial prototype as depicted in Fig.
- Additionally, a live demonstration exhibiting the system capabilities on autonomous driving was also carried out during the IEEE Conference on Intelligent Vehicles 2002, in a private circuit located at Satory , France.
2.1. Region of Interest
- Nonetheless, the use of temporal filtering techniques (as described in the following sections) allows to obtain finer resolution estimations.
- The probability of finding the most relevant road features is assured to be high by making use of a priori knowledge on the road shape, according to the parabolic road model proposed.
- Thus, in most cases the region of interest is reduced to some portion of image surrounding the road edges estimated in the previous iteration of the algorithm.
- This is a valid assumption for road tracking applications heavily relying on the detection of lane markers that represent the road edges.
- This restriction permits to remove nonrelevant elements from the image such as the sky, trees, buildings, etc.
2.2. Road Features
- The combined use of color and shape restrictions provides the essential information required to drive on non structured roads.
- This makes highly recommendable the use of a color space where a clear separation between intensity and color information can be established.
- Hue represents the impression related to the predominant wavelength in the perceived color stimulus.
- This could save some computing time by avoiding going through the trigonometry.
- 1990; Rodriguez et al., 1998), the HSI color space has exhibited superior performance in image segmentation problems as demonstrated in Ikonomakis et al. (2000).
2.3. Road Model
- The use of a road model eases the reconstruction of the road geometry and permits to filter the data computed during the features searching process.
- More concretely, the use of parabolic functions to model the projection of the road edges onto the image plane has been proposed and successfully tested in previous works (Schneiderman and Nashman, 1994).
- A second order polynomial model has only three adjustable coefficients, also known as Simplicity.
- Discontinuities in the road model are only encountered in road intersections and, particularly, in crossroads.
- The adjustable parameters of the several parabolic functions are continuously updated at each iteration of the algorithm using a well known least squares estimator, as will be described later.
2.4. Road Segmentation
- Image segmentation must be carried out by exploiting the cylindrical distribution of color features in the HSI color space, bearing in mind that the separation between road and no road color characteristics is nonlinear.
- According to this, pixels are divided into chromatic and achromatic as proposed in Ikonomakis et al. (2000).
- This turns the segmentation stage into a position dependant process.
- Thus, for pixels clearly located out of the road trajectory, the chromatic and luminance distances to the road pattern color features should be very small in order to effectively be segmented as part of the road.
- Seven a priori road models are utilized for this purpose, as depicted in Fig.
2.5. Handling Shadows and Brightness
- Shadows and brightness on the road are admittedly the greatest difficulty in vision based systems operating in outdoor environments (Bertozzi and Broggi, 1998).
- B ≥ 1 3 I ≤ Iroad,avg − 2 · σroad (15) where b stands for the normalized blue component; Iroad,avg represents the average intensity value of all road pixels, and σroad is the standard deviation of the intensity distribution of road pixels.
- This technique permits to enhance the road segmentation in presence of shadows, and remarkably contributes to improve the robustness of the color adaptation process, particularly in stretches of road largely covered by shadows.
- Analytically the condition is formulated in Eq. (16).
- The improvement achieved by attenuating both brightness and shadows, as described, permits to handle real images in real and complex situations with an extraordinary high performance, becoming an outstanding point of this work.
2.6. Estimation of Road Edges and Width
- The estimation of the skeleton lines of the road and its edges is carried out basing on parabolic functions, as previously described.
- These polynomial functions are the basis to obtain the lateral and orientation error of the vehicle with respect to the center of the lane.
- On the other hand, ŷl(0), ŷc(0), ŷr (0) are the initial estimations for the left edge, right edge, and skeleton lines of the road, respectively, while yli , yri,yci stand for the left edge, right edge, and skeleton lines of basic pattern i .
2.6.2. Estimation of the Skeleton Lines of the Road.
- The skeleton lines of the road at current time instant, ŷc(t), is estimated based on the segmented low resolution image and the previously estimated road trajectory, ŷc(t − 1).
- Thus, the estimation is realized in three steps as described below.
- The estimation of road edges is realized using the same filtering technique described in the previous section.
- For each line in the area of interest, the closest measurements to the middle of the left edge validation area, defined by ŷc(t)−Ŵ (t −1)/2, and right edge validation area, defined by ŷc(t) +.
- An individual road width measure wi is obtained for each line in the region of interest, by computing the difference between the left and right edges (ŷl(t)|x=xi and ŷr (t)|x=xi , respectively) as expressed in Eq. (20).
2.7. Road Color Features Update
- After completing the road edges and width estimation process, the HSI color features of the road pattern are consequently updated so as to account for changes in road appearance and illumination.
- Intuitively, pixels close to the skeleton lines of the road present color features that highly represent the road color pattern.
- Obviously, the selected pixels are only validated if they have been segmented as road pixels at the current iteration.
- The adaptation process described in this section proves to be crucial in practice to keep the segmentation algorithm under stable performance upon illumination changing conditions and color varying asphalt.
- The complete road tracking scheme is graphically summarised in the flow diagram depicted in Fig. 24.
2.8. Discussion of the Method
- The global objective of this section is to put the road tracking algorithm under test in varied real circumstances.
- As appreciated from observation of Fig. 25, the road edges can be neatly distinguished in the segmented image, allowing a clear estimation in real experiments.
- The results obtained in presence of other vehicles parked on the left hand side of the road are illustrated in Fig. 27(b).
- All pixels in the image tend to have similar intensity values, and thus, color differences in the HSI chromatic plane become crucial for segmentation purposes.
- This situation derives in the appearance of dark spots on the road due to wet areas, as depicted in Fig. 31.
3. Implementation and Results
- The complete navigation system described in the previous sections has been implemented on the so-called Babieca prototype vehicle, depicted in Fig. 1, that has been modified to allow for automatic velocity and steering control at a maximum speed of 90 km/h, using the non linear control law developed in Sotelo (2001).
- As stated in Section 1, a live demonstration exhibiting the system capabilities on autonomous driving was also carried out during the IEEE Conference on Intelligent Vehicles 2002, in a private circuit located at Satory , France.
- In order to complete the graphical results depicted in the previous sections, and to illustrate the global behavior of the complete navigation system implemented on Babieca, some general results are shown next.
- During the tests, the reference vehicle velocity is assumed to be kept constant by the velocity controller.
- To complete these results a wide set of video files demonstrating the operational performance of the system in real tests can be retrieved from ftp://www. depeca.uah.es/pub/vision.
4. Conclusions
- The road segmentation algorithm based on the HSI color space and 2D-spatial constraints, as described in this work, has successfully proved to provide correct estimations for the edges and width of non-structured roads, i.e., roads without lane markers.
- The practical results discussed above also support the validity of the method for different environmental and weather conditions, as demonstrated so far.
- Otherwise, there will be long shadows but the system performs well.
- The most remarkable feature of the road tracking scheme described in this work is its ability to correctly deal with non-structured roads by performing a non-supervised color-based road segmentation process.
- Nonetheless, a lot of work still remains to be done until a completely robust and reliable autonomous system can be fully deployed in real conditions.
Did you find this useful? Give us your feedback
Citations
327 citations
Cites background or methods from "A Color Vision-Based Lane Tracking ..."
...Thus, additional constraints must be included to improve their performance, e.g., in [ 13 ], Sotelo et al. use road shape restrictions and ad hoc postprocessing to recover undetected shadowed areas....
[...]
...For comparison, the image sequence SRD is processed using an HSI-based road-detection algorithm inspired in [7] and [ 13 ]....
[...]
...Other works in the field [6], [ 13 ] follow similar assumptions....
[...]
194 citations
184 citations
109 citations
Cites background or methods from "A Color Vision-Based Lane Tracking ..."
...The last two algorithms are photometric–based approaches: the illuminant–invariant algorithm in [18] and the HSI based algorithm proposed in [1] and used in [5]....
[...]
...The two most popular color spaces, that have proved to be robust to minor illuminant changes, are HSV [1, 5] and normalized RGB [2]....
[...]
...In general, vision–based methods use low-level features for road detection [1, 2, 3, 4]....
[...]
...Algorithms based on transformed color spaces (i.e., HSV [5, 1] or rg [2]) are, to a certain degree still sensitive to lighting variations such as strong shadows and highlights....
[...]
..., HSV [5, 1] or rg [2]) are, to a certain degree still sensitive to lighting variations such as strong shadows and highlights....
[...]
93 citations
References
55 citations
"A Color Vision-Based Lane Tracking ..." refers background or methods in this paper
...According to this, pixels are divided into chromatic and achromatic as proposed in Ikonomakis et al. (2000)....
[...]
...Although the RGB color space has been successfully used in previous works dealing with road segmentation (Thorpe, 1990; Rodriguez et al., 1998), the HSI color space has exhibited superior performance in image segmentation problems as demonstrated in Ikonomakis et al. (2000)....
[...]
...A human can easily recognize basic color attributes: intensity (luminance or brightness), hue or color, and saturation (Ikonomakis et al., 2000)....
[...]
48 citations
"A Color Vision-Based Lane Tracking ..." refers background in this paper
...…Mellon University, the impressive unmanned vehicles developed during the last decade by the research groups at the UBM (Dickmanns et al., 1994; Lutzeler and Dickmanns, 1998) and Daimler-Benz (Franke et al., 1998), or the GOLD system (Bertozzi and Broggi, 1998; Broggi et al., 1999)…...
[...]
...Likewise, automatic cooperative driving of vehicle fleets involved in the transportation of heavy loads can lead to notable industrial cost reductions....
[...]
42 citations
"A Color Vision-Based Lane Tracking ..." refers methods in this paper
...More concretely, the use of parabolic functions to model the projection of the road edges onto the image plane has been proposed and successfully tested in previous works (Schneiderman and Nashman, 1994)....
[...]
33 citations
"A Color Vision-Based Lane Tracking ..." refers background or methods in this paper
...Although properly parameterized edge-based methods can also be used to detect the road edges (Lutzeler and Dickmanns, 2000), the consideration of color properties of the complete road surface, as proposed in this paper, serves also for vehicle detection reinforcement....
[...]
...Likewise, another similar system can be found in Lutzeler and Dickmanns (2000) and Gregor et al. (2002), where a real autonomous system for Intelligent Navigation in a network of unmarked roads and intersections is designed and implemented using edge detectors for lane tracking....
[...]
...Likewise, automatic cooperative driving of vehicle fleets involved in the transportation of heavy loads can lead to notable industrial cost reductions....
[...]
19 citations
"A Color Vision-Based Lane Tracking ..." refers background in this paper
...Likewise, automatic cooperative driving of vehicle fleets involved in the transportation of heavy loads can lead to notable industrial cost reductions....
[...]
...After that, the development of a vision-based system (Sotelo et al., 2001; De Pedro et al., 2001) for Autonomous Vehicle Driving on unmarked roads was undertaken until reaching the results presented in this paper....
[...]