This paper provides a review of the literature in on-road vision-based vehicle detection, tracking, and behavior understanding. Over the past decade, vision-based surround perception has progressed from its infancy into maturity. We provide a survey of recent works in the literature, placing vision-based vehicle detection in the context of sensor-based on-road surround analysis. We detail advances in vehicle detection, discussing monocular, stereo vision, and active sensor-vision fusion for on-road vehicle detection. We discuss vision-based vehicle tracking in the monocular and stereo-vision domains, analyzing filtering, estimation, and dynamical models. We discuss the nascent branch of intelligent vehicles research concerned with utilizing spatiotemporal measurements, trajectories, and various features to characterize on-road behavior. We provide a discussion on the state of the art, detail common performance metrics and benchmarks, and provide perspective on future research directions in the field.

https://cvrr.ucsd.edu/publications/2013/SayananTrivedi_IEEETITS2013_VehicleSurvey.pdf

Looking at Vehicles on the Road: A Survey of Vision-Based Vehicle Detection, Tracking, and Behavior Analysis

The problem of road or lane perception is a crucial enabler for advanced driver assistance systems. As such, it has been an active field of research for the past two decades with considerable progress made in the past few years. The problem was confronted under various scenarios, with different task definitions, leading to usage of diverse sensing modalities and approaches. In this paper we survey the approaches and the algorithmic techniques devised for the various modalities over the last 5 years. We present a generic break down of the problem into its functional building blocks and elaborate the wide range of proposed methods within this scheme. For each functional block, we describe the possible implementations suggested and analyze their underlying assumptions. While impressive advancements were demonstrated at limited scenarios, inspection into the needs of next generation systems reveals significant gaps. We identify these gaps and suggest research directions that may bridge them.

Recent progress in road and lane detection: a survey

Aspects of the invention relate generally to autonomous vehicles. Specifically, the features described may be used alone or in combination in order to improve the safety, use, driver experience, and performance of these vehicles.

System and method for predicting behaviors of detected objects

System and method for conveying data between vehicles includes a data generating system arranged on a first vehicle to obtain or generate information about the first vehicle or conditions around the first vehicle, a first communications system arranged on the first vehicle and coupled to the data generating system for communicating with a wireless Internet service provider (ISP) and a second communications system arranged on a second vehicle and communicating with an ISP. The first communications system enables the information obtained or generated by the data generating system to be transmitted via the Internet to the second communications system. The first and second communications systems can communicate with the same ISP or different ISPs linked to one another.

Intra-vehicle information conveyance system and method

Autonomous vehicle navigation is challenging since various types of road scenarios in real urban environments have to be considered, particularly when only perception sensors are used, without position information. This paper presents a novel real-time optimal-drivable-region and lane detection system for autonomous driving based on the fusion of light detection and ranging (LIDAR) and vision data. Our system uses a multisensory scheme to cover the most drivable areas in front of a vehicle. We propose a feature-level fusion method for the LIDAR and vision data and an optimal selection strategy for detecting the best drivable region. Then, a conditional lane detection algorithm is selectively executed depending on the automatic classification of the optimal drivable region. Our system successfully handles both structured and unstructured roads. The results of several experiments are provided to demonstrate the reliability, effectiveness, and robustness of the system.

A Sensor-Fusion Drivable-Region and Lane-Detection System for Autonomous Vehicle Navigation in Challenging Road Scenarios

This paper presents an approach to the lane recognition using on-vehicle LIDAR. It detests the objects by 2-dimensional scanning and collects the range and reflectivity data in each scanning direction. We developed the lane recognition algorithm with these data, in which the lane curvature, yaw angle and offset are calculated by using the Hough transformation, and the lane width is calculated by statistical procedure. Next the lane marks are tracked by the Extended Kalman Filter. Then we test the performance of the lane recognition and the good results are achieved. Finally we show the result of the road environment recognition applying the lane recognition by LIDAR.

Lane Recognition Using On-vehicle LIDAR

In this paper, we present the road environment recognition algorithm using on-vehicle LIDAR. Although our 2-dimensional scanning and high-resolution LIDAR can detect the forward vehicle, other objects and the lane mark at the same time, we have only applied the LIDAR to ACC system by recognizing only vehicles. To pursue the potential of the LIDAR, DENSO has developed the road environment recognition algorithm using only the LIDAR as a forward object detection sensor with compensation of the ego-vehicle motion. As a result, we constructed a model-based algorithm, which could recognize various objects on the absolute coordinate system. Moreover we show the possibility of the coordination with LIDAR and map or GPS for high accurate navigation.

Road Environment Recognition Using On-vehicle LIDAR

Lane recognition is a function which is needed for a variety of driver assistance systems. For example Lane Departure Warning and Lane Keeping rely on information provided by a lane estimation algorithm. One important step of the lane estimation procedure is the extraction of measurements or detections which can be used to estimate the shape of the road or lane. These detections are generated by white lane markers or the road border itself. Lane estimation has for many years been under heavy development using a gray scale camera. Passive camera based systems can be degraded on its performance under certain circumstances, e.g. at dynamic changes of ambient brightness. The fusion with an active sensor can here increase the robustness of these systems significantly. In this paper an approach is presented to detect lane marks using an active light detection and ranging device (lidar). It can be shown that high reflective lane marks can be reliably detected. A polar Lane Detector Grid is used to combine the distance and intensity measurement channel of the lidar sensor.

Multi-channel lidar processing for lane detection and estimation

Automatic acceleration/deceleration control for an automotive vehicle is provided which may be used in an automatic deceleration system and a vehicle-to-obstacle distance control system. The automatic acceleration/deceleration control determines a brake pressure control command when decelerating the vehicle based on a deceleration difference between a target deceleration and an actual deceleration of the vehicle and adjusts the brake pressure acting on a wheel of the vehicle in response to the brake pressure control command to decelerate the vehicle so that the actual deceleration agrees with the target deceleration. The automatic acceleration/deceleration control further corrects the brake pressure control command by modifying the deceleration difference for compensating for a time delay between output of the brake pressure control command and a time when the vehicle is actually decelerated.

Automatic deceleration control system, vehicle-to-obstacle distance control system, and system program storage medium for vehicle

This paper presents a method for pedestrian tracking in urban environments using in-vehicle multilayer laser lidar (MLLR). The MLLR that we developed irradiates the laser in six scanning planes by a polygon mirror mechanism, and thus objects with height are observed with the plural scanning planes. MLLR outputs are modified by GPS data and are mapped onto a grid map. Pedestrians are found based on the occupancy grid method, and they are tracked via Kalman filter in conjunction with global nearest neighboring (GNN) based data association. A track management method improves tracking accuracy in real worlds. Our tracking algorithm works well in a low-performance computer environment. The experimental results in different scenarios such as intersection and on the community road validate the proposed method.

Kiyokazu Takagi

Papers

Lane Recognition Using On-vehicle LIDAR

Road Environment Recognition Using On-vehicle LIDAR

Multi-channel lidar processing for lane detection and estimation

Automatic deceleration control system, vehicle-to-obstacle distance control system, and system program storage medium for vehicle

Multilayer lidar-based pedestrian tracking in urban environments