Robust Lane Detection for Complicated Road Environment Based on Normal Map
TL;DR: This work improves an adaptive threshold segmentation method and denoising operations to enhance the lane markings and generates a normal map by using the depth information and extracting a segmented road pavement without vehicles and buildings based on the normal map.
Abstract: Detection of road or lane is indispensable for the environmental perception of advanced driver assistance systems. It has been an active field of research with a wide application prospect. However, due to the complex illumination and interferences, such as vehicles and shadows in the real driving environment, lane detection is still a challenging task today. To address these issues, a robust method for road segmentation and lane detection based on a normal map is proposed. The first step of this approach is to generate the normal map by using the depth information and then extract a segmented road pavement without vehicles and buildings based on the normal map. Second, we improve an adaptive threshold segmentation method and denoising operations to enhance the lane markings. Third, the combination of Hough transform and vanishing point makes it more accurate to determine the starting points of host lanes, and then the lanes in the following image sequence can be detected in the adaptive region of interest. Compared with the state-of-the-art methods, the experimental results on the data sets in two countries demonstrate that our approach produces more credible performance under various light conditions or dense traffic.
Citations
More filters
TL;DR: This work introduces a new geometry-guided lane anchor representation in a new coordinate frame and applies a specific geometric transformation to directly calculate real 3D lane points from the network output and presents a scalable two-stage framework that decouples the learning of image segmentation subnetwork and geometry encoding subnetwork.
Abstract: We present a generalized and scalable method, called Gen-LaneNet, to detect 3D lanes from a single image. The method, inspired by the latest state-of-the-art 3D-LaneNet, is a unified framework solving image encoding, spatial transform of features and 3D lane prediction in a single network. However, we propose unique designs for Gen-LaneNet in two folds. First, we introduce a new geometry-guided lane anchor representation in a new coordinate frame and apply a specific geometric transformation to directly calculate real 3D lane points from the network output. We demonstrate that aligning the lane points with the underlying top-view features in the new coordinate frame is critical towards a generalized method in handling unfamiliar scenes. Second, we present a scalable two-stage framework that decouples the learning of image segmentation subnetwork and geometry encoding subnetwork. Compared to 3D-LaneNet, the proposed Gen-LaneNet drastically reduces the amount of 3D lane labels required to achieve a robust solution in real-world application. Moreover, we release a new synthetic dataset and its construction strategy to encourage the development and evaluation of 3D lane detection methods. In experiments, we conduct extensive ablation study to substantiate the proposed Gen-LaneNet significantly outperforms 3D-LaneNet in average precision(AP) and F-score.
48 citations
TL;DR: An overview oflane-level road network generation techniques for the lane-level maps of autonomous vehicles with on-board systems, including the representation and generation of lane- level road networks is presented.
Abstract: Autonomous driving is experiencing rapid development. A lane-level map is essential for autonomous driving, and a lane-level road network is a fundamental part of a lane-level map. A large amount of research has been performed on lane-level road network generation based on various on-board systems. However, there is a lack of analysis and summaries with regards to previous work. This paper presents an overview of lane-level road network generation techniques for the lane-level maps of autonomous vehicles with on-board systems, including the representation and generation of lane-level road networks. First, sensors for lane-level road network data collection are discussed. Then, an overview of the lane-level road geometry extraction methods and mathematical modeling of a lane-level road network is presented. The methodologies, advantages, limitations, and summaries of the two parts are analyzed individually. Next, the classic logic formats of a lane-level road network are discussed. Finally, the survey summarizes the results of the review.
24 citations
Cites methods from "Robust Lane Detection for Complicat..."
...To establish the mathematical model of a structure, a Hough transform [77] was used in the pre‐ extraction before curve fitting....
[...]
TL;DR: In this paper, a vision-based road lane detection technique plays a key role in driver assistance system, while existing lane recognition algorithms demonstrated over 90% detection rate, the validation test was...
Abstract: The vision-based road lane detection technique plays a key role in driver assistance system. While existing lane recognition algorithms demonstrated over 90% detection rate, the validation test was...
20 citations
TL;DR: A deep convolutional neural network based on FCN network is used for lane boundary feature extraction, and the neural network can be used to classify the lane images at the pixel level to solve the problem of the poor robustness for extracting the multi-lane marking.
Abstract: In order to solve the problem of the poor robustness for extracting the multi-lane marking, a multi-lane detection algorithm based on deep convolutional neural network is proposed. Since the results of the feature extraction of lane boundary were affected by various factors, a deep convolutional neural network based on FCN network is used for lane boundary feature extraction, and the neural network can be used to classify the lane images at the pixel level. Then the network parameters are trained on the public Tusimple data set and evaluated at Caltech Lanes data set. The last part combines a linear model with a curved model to realize the establishment of the lane marking equation, Hough transform is used to determine the fit interval and the least square method is used to fit lane marking. The experimental results show that the average accuracy of the algorithm for identifying lane on the Tusimple data set is 98.74%, and the accuracy rate on the Caltech Lanes data set is 96.29%.
19 citations
Cites methods from "Robust Lane Detection for Complicat..."
...Monocular cameras were used in [2]–[4] as a road information acquisition sensor, and binocular cameras were used in [5]–[8] generally combine the detection of parallax maps and vanishing points to achieve drivable road detection....
[...]
TL;DR: Experimental results show that DBMW outperforms existing methods and can significantly improve the detection accuracy and false alarm rates of dangerous driving behaviors.
Abstract: In this paper, we design a driver behavior monitoring and warning (DBMW) framework to detect dangerous driving for enhancing road safety through the Internet of Vehicles (IoV). The designed DBMW framework applies onboard image sensors and wearable devices to detect the deviation degree of vehicles and trace the head motion of drivers, respectively. According to our review of relevant research, DBMW is the first framework for driver behavior monitoring and warning that provides the following features: 1) DBMW can keep recognizing the located lane lines and estimating the power spectral density of lane deviation for a vehicle through image sensors, 2) DBMW can keep monitoring driver behaviors and measuring the anomaly level of a driver through wearable devices, and 3) DBMW can instantly send the warning messages of potential dangerous driving to neighboring vehicles and nearby pedestrians through IoV communications as necessary, which makes vehicles and pedestrians be aware of the existence of surrounding dangerous drivers in advance to keep alerting and avoid potential accidents/collisions. In particular, the prototype consisting of an Android-based sensing unit and an Arduino-based wearable device is implemented to verify the feasibility and superiority of DBMW. Experimental results show that DBMW outperforms existing methods and can significantly improve the detection accuracy and false alarm rates of dangerous driving behaviors.
16 citations
References
More filters
07 Jun 2015
TL;DR: The key insight is to build “fully convolutional” networks that take input of arbitrary size and produce correspondingly-sized output with efficient inference and learning.
Abstract: Convolutional networks are powerful visual models that yield hierarchies of features. We show that convolutional networks by themselves, trained end-to-end, pixels-to-pixels, exceed the state-of-the-art in semantic segmentation. Our key insight is to build “fully convolutional” networks that take input of arbitrary size and produce correspondingly-sized output with efficient inference and learning. We define and detail the space of fully convolutional networks, explain their application to spatially dense prediction tasks, and draw connections to prior models. We adapt contemporary classification networks (AlexNet [20], the VGG net [31], and GoogLeNet [32]) into fully convolutional networks and transfer their learned representations by fine-tuning [3] to the segmentation task. We then define a skip architecture that combines semantic information from a deep, coarse layer with appearance information from a shallow, fine layer to produce accurate and detailed segmentations. Our fully convolutional network achieves state-of-the-art segmentation of PASCAL VOC (20% relative improvement to 62.2% mean IU on 2012), NYUDv2, and SIFT Flow, while inference takes less than one fifth of a second for a typical image.
28,225 citations
"Robust Lane Detection for Complicat..." refers background or methods in this paper
...As a deep learning method, FCN requires a large amount of data to train 49686 VOLUME 6, 2018 a powerful model....
[...]
..., IPM + hyperbolic model [10], superparticle method [16], lane detection with two-stage feature extraction (LDTFE) method [26], and a state-of-the-art fully convolutional networks (FCN) method [32]....
[...]
...(a1)∼(a10) are the results of IPM + hyperbolic model [10], cyan lines refer to the detected lanes; (b1)∼(b10) are the results of superparticle [16], blue points refer to the detected lanes; (c1)∼(c10) are the results of LDTFE [26], magenta lines refer to the detected lanes; (d1)∼(d10) are the results of FCN [32], green area refers to the host lane; (e1)∼(e10) are the results of the proposed method, red lines refer to the detected lanes, and green area refers to the host lane....
[...]
...FCN method [32], a fully convolutional network is used for semantic segmentation of the host lane....
[...]
...In the FCN method [32], a fully convolutional network is used for semantic segmentation of the host lane....
[...]
01 Jan 1979
1,064 citations
"Robust Lane Detection for Complicat..." refers methods in this paper
...One is Otsu’s method [30] which is a kind of global threshold method....
[...]
...(a1)∼(a4) are the original images; (b1)∼(b4) are the results of road segmentation; (c1)∼(c4) are the results of binarization with Otsu’s method [30]; (d1)∼(d4) are the results of proposed threshold segmentation method; (e1)∼(e4) are the results after removing noise....
[...]
01 Apr 2014
TL;DR: This paper presents a generic break down of the problem of road or lane perception into its functional building blocks and elaborate the wide range of proposed methods within this scheme.
Abstract: The problem of road or lane perception is a crucial enabler for advanced driver assistance systems. As such, it has been an active field of research for the past two decades with considerable progress made in the past few years. The problem was confronted under various scenarios, with different task definitions, leading to usage of diverse sensing modalities and approaches. In this paper we survey the approaches and the algorithmic techniques devised for the various modalities over the last 5 years. We present a generic break down of the problem into its functional building blocks and elaborate the wide range of proposed methods within this scheme. For each functional block, we describe the possible implementations suggested and analyze their underlying assumptions. While impressive advancements were demonstrated at limited scenarios, inspection into the needs of next generation systems reveals significant gaps. We identify these gaps and suggest research directions that may bridge them.
735 citations
"Robust Lane Detection for Complicat..." refers background in this paper
...So advanced driver assistance systems (ADAS) [1], [2] which are designed to help drivers in driving process emerge as the times require....
[...]
...In the past two decades, researchers have made considerable progress in the vision-based approaches [2], [3], while facing several major challenges, especially in attaining robustness under complex lighting conditions and dense traffic....
[...]
24 Oct 2014
TL;DR: This contribution provides a review of fundamental goals, development and future perspectives of driver assistance systems, and examines the progress incented by the use of exteroceptive sensors such as radar, video, or lidar in automated driving in urban traffic and in cooperative driving.
Abstract: This contribution provides a review of fundamental goals, development and future perspectives of driver assistance systems. Mobility is a fundamental desire of mankind. Virtually any society strives for safe and efficient mobility at low ecological and economic costs. Nevertheless, its technical implementation significantly differs among societies, depending on their culture and their degree of industrialization. A potential evolutionary roadmap for driver assistance systems is discussed. Emerging from systems based on proprioceptive sensors, such as ABS or ESC, we review the progress incented by the use of exteroceptive sensors such as radar, video, or lidar. While the ultimate goal of automated and cooperative traffic still remains a vision of the future, intermediate steps towards that aim can be realized through systems that mitigate or avoid collisions in selected driving situations. Research extends the state-of-the-art in automated driving in urban traffic and in cooperative driving, the latter addressing communication and collaboration between different vehicles, as well as cooperative vehicle operation by its driver and its machine intelligence. These steps are considered important for the interim period, until reliable unsupervised automated driving for all conceivable traffic situations becomes available. The prospective evolution of driver assistance systems will be stimulated by several technological, societal and market trends. The paper closes with a view on current research fields.
716 citations
"Robust Lane Detection for Complicat..." refers background in this paper
...So advanced driver assistance systems (ADAS) [1], [2] which are designed to help drivers in driving process emerge as the times require....
[...]
...His research interests include computer vision, image processing, and ADAS....
[...]
...Lane-mark detection is one of the most important parts of ADAS and autonomous driving cars, and it’s also a precondition for lane departure warning (LDW) [3]–[5]....
[...]
04 Jun 2008
TL;DR: In this paper, a robust and real-time approach to lane marker detection in urban streets is presented, which is based on generating a top view of the road, filtering using selective oriented Gaussian filters, using RANSAC line fitting to give initial guesses to a new and fast RANAC algorithm for fitting Bezier Splines, which was then followed by a post-processing step.
Abstract: We present a robust and real time approach to lane marker detection in urban streets. It is based on generating a top view of the road, filtering using selective oriented Gaussian filters, using RANSAC line fitting to give initial guesses to a new and fast RANSAC algorithm for fitting Bezier Splines, which is then followed by a post-processing step. Our algorithm can detect all lanes in still images of the street in various conditions, while operating at a rate of 50 Hz and achieving comparable results to previous techniques.
672 citations