Convolutional neural networks (CNNs) are usually built by stacking convolutional operations layer-by-layer. Although CNN has shown strong capability to extract semantics from raw pixels, its capacity to capture spatial relationships of pixels across rows and columns of an image is not fully explored. These relationships are important to learn semantic objects with strong shape priors but weak appearance coherences, such as traffic lanes, which are often occluded or not even painted on the road surface as shown in Fig. 1 (a). In this paper, we propose Spatial CNN (SCNN), which generalizes traditional deep layer-by-layer convolutions to slice-by-slice convolutions within feature maps, thus enabling message passings between pixels across rows and columns in a layer. Such SCNN is particular suitable for long continuous shape structure or large objects, with strong spatial relationship but less appearance clues, such as traffic lanes, poles, and wall. We apply SCNN on a newly released very challenging traffic lane detection dataset and Cityscapse dataset. The results show that SCNN could learn the spatial relationship for structure output and significantly improves the performance. We show that SCNN outperforms the recurrent neural network (RNN) based ReNet and MRF+CNN (MRFNet) in the lane detection dataset by 8.7% and 4.6% respectively. Moreover, our SCNN won the 1st place on the TuSimple Benchmark Lane Detection Challenge, with an accuracy of 96.53%.

https://www.aaai.org/ocs/index.php/AAAI/AAAI18/paper/download/16802/16322

Spatial as deep: Spatial CNN for traffic scene understanding

Autonomous vehicles are expected to play a key role in the future of urban transportation systems, as they offer potential for additional safety, increased productivity, greater accessibility, better road efﬁciency, and positive impact on the environment. Research in autonomous systems has seen dramatic advances in recent years, due to the increases in available computing power and reduced cost in sensing and computing technologies, resulting in maturing technological readiness level of fully autonomous vehicles. The objective of this paper is to provide a general overview of the recent developments in the realm of autonomous vehicle software systems. Fundamental components of autonomous vehicle software are reviewed, and recent developments in each area are discussed.

Perception, Planning, Control, and Coordination for Autonomous Vehicles

This paper presents a comprehensive literature review on environment perception for intelligent vehicles. The state-of-the-art algorithms and modeling methods for intelligent vehicles are given, with a summary of their pros and cons. A special attention is paid to methods for lane and road detection, traffic sign recognition, vehicle tracking, behavior analysis, and scene understanding. In addition, we provide information about datasets, common performance analysis, and perspectives on future research directions in this area.

/pdf/overview-of-environment-perception-for-intelligent-vehicles-7s8l9oth23.pdf

Overview of Environment Perception for Intelligent Vehicles

A survey on Visual-Based Localization: On the benefit of heterogeneous data

Feature detection, description and matching are essential components of various computer vision applications, thus they have received a considerable attention in the last decades. Several feature detectors and descriptors have been proposed in the literature with a variety of definitions for what kind of points in an image is potentially interesting (i.e., a distinctive attribute). This chapter introduces basic notation and mathematical concepts for detecting and describing image features. Then, it discusses properties of perfect features and gives an overview of various existing detection and description methods. Furthermore, it explains some approaches to feature matching. Finally, the chapter discusses the most used techniques for performance evaluation of detection and description algorithms.

Image Features Detection, Description and Matching

Invariant property of lane color under various illuminations is utilized for lane detection.Computational complexity is reduced using vanishing point detection and adaptive ROI.Datasets for evaluation include various environments from several devices.Simulation demo demonstrate fast and powerful performance for real-time applications. Lane detection is an important element in improving driving safety. In this paper, we propose a real-time and illumination invariant lane detection method for lane departure warning system. The proposed method works well in various illumination conditions such as in bad weather conditions and at night time. It includes three major components: First, we detect a vanishing point based on a voting map and define an adaptive region of interest (ROI) to reduce computational complexity. Second, we utilize the distinct property of lane colors to achieve illumination invariant lane marker candidate detection. Finally, we find the main lane using a clustering method from the lane marker candidates. In case of lane departure situation, our system sends driver alarm signal. Experimental results show satisfactory performance with an average detection rate of 93% under various illumination conditions. Moreover, the overall process takes only 33ms per frame.

Real-time illumination invariant lane detection for lane departure warning system

Lane detection is important in many advanced driver-assistance systems (ADAS). Vision-based lane detection algorithms are widely used and generally use gradient information as a lane feature. However, gradient values between lanes and roads vary with illumination change, which degrades the performance of lane detection systems. In this paper, we propose a gradient-enhancing conversion method for illumination-robust lane detection. Our proposed gradient-enhancing conversion method produces a new gray-level image from an RGB color image based on linear discriminant analysis. The converted images have large gradients at lane boundaries. To deal with illumination changes, the gray-level conversion vector is dynamically updated. In addition, we propose a novel lane detection algorithm, which uses the proposed conversion method, adaptive Canny edge detector, Hough transform, and curve model fitting method. We performed several experiments in various illumination environments and confirmed that the gradient is maximized at lane boundaries on the road. The detection rate of the proposed lane detection algorithm averages 96% and is greater than 93% in very poor environments.

Gradient-Enhancing Conversion for Illumination-Robust Lane Detection

A vision based rear obstacle detection system is one of the most essential technologies, which can be used in many applications such as a parking assistance systems and intelligent vehicles. Stereo matching based obstacle detection methods have two inherent limitations, including sensitivity to illumination variation and high computational complexity. To overcome these problems, we present a hierarchical census transform (HCT)-based stereo matching method, and proposes a real-time rear obstacle detection system using fisheye stereo cameras. It includes two major components: firstly, we use a hierarchical approach to improve computational efficiency and reduce dependency on a matching window size. Computation time can also be accelerated with compute unified device architecture (CUDA) implementation because we designed the proposed method for parallel processing. Secondly, our method provides more accurate depth information, especially for moving objects, through cost aggregation using color and motion information. Experimental results show satisfactory performance under various real parking environments. The performance of the proposed method is superior to those of the conventional methods in terms of runtime and accuracy of depth estimation. Moreover, correct detection rate of the proposed system is 12.51% better than those of other systems, while its false and miss detection rates are 11.09% and 12.51% lower than those of other detection systems.

Rear obstacle detection system with fisheye stereo camera using HCT

Exact order based feature descriptor for illumination robust image matching

HightlightsUtilizing three features such as disparity, super pixel segments and pixel-wise gradient.Computing the reliability of disparity from super pixel segments and pixel-wise gradient.Developing voting map to reduce time complexity of initial obstacle region.Superior performance with erroneous disparity information and in complex environments. A vision based real-time rear obstacle detection system is one of the most essential technologies, which can be used in many applications such as a parking assistance systems and intelligent vehicles. Although disparity is a useful feature for detecting obstacles, estimating a correct disparity map is a hard problem due to the matching ambiguity and noise sensitivity, especially in homogeneous regions. To overcome these problems, we leverage reliable disparities only for obstacle detection. A reliability factor is introduced to measure an inhomogeneity of the regions quantitatively. It is computed at each superpixel to consider the noise sensitivity of pixel-wise gradients and to assign similar reliability value within a same object. It includes two major components: firstly, In a feature extraction and combining stage, we extract three features from stereo images such as disparity, superpixel segments and pixel-wise gradient and compute the reliability of disparity from superpixel segments and the pixel-wise gradient. Secondly, In an obstacle detection stage, a disparity feature with reliability votes for localizing obstacles and dominant candidates in voting map are selected as initial obstacle region. The initial obstacle regions are expanded into their neighbor superpixels based on CIELAB color similarity and distance similarity between superpixels. Experimental results show satisfactory performance under various real parking environments. Its detection rate is at least 4% higher than those of other existing methods, and its false detection rate is more than 10% lower and thus, can be used for parking assistance system.

Hunjae Yoo

Papers

Real-time illumination invariant lane detection for lane departure warning system

Gradient-Enhancing Conversion for Illumination-Robust Lane Detection

Rear obstacle detection system with fisheye stereo camera using HCT

Exact order based feature descriptor for illumination robust image matching

Real-time rear obstacle detection using reliable disparity for driver assistance