This paper provides a review of the literature in on-road vision-based vehicle detection, tracking, and behavior understanding. Over the past decade, vision-based surround perception has progressed from its infancy into maturity. We provide a survey of recent works in the literature, placing vision-based vehicle detection in the context of sensor-based on-road surround analysis. We detail advances in vehicle detection, discussing monocular, stereo vision, and active sensor-vision fusion for on-road vehicle detection. We discuss vision-based vehicle tracking in the monocular and stereo-vision domains, analyzing filtering, estimation, and dynamical models. We discuss the nascent branch of intelligent vehicles research concerned with utilizing spatiotemporal measurements, trajectories, and various features to characterize on-road behavior. We provide a discussion on the state of the art, detail common performance metrics and benchmarks, and provide perspective on future research directions in the field.

https://cvrr.ucsd.edu/publications/2013/SayananTrivedi_IEEETITS2013_VehicleSurvey.pdf

Looking at Vehicles on the Road: A Survey of Vision-Based Vehicle Detection, Tracking, and Behavior Analysis

In this paper, we present a novel probabilistic compact representation of the on-road environment, i.e., the dynamic probabilistic drivability map (DPDM), and demonstrate its utility for predictive lane change and merge (LCM) driver assistance during highway and urban driving. The DPDM is a flexible representation and readily accepts data from a variety of sensor modalities to represent the on-road environment as a spatially coded data structure, encapsulating spatial, dynamic, and legal information. Using the DPDM, we develop a general predictive system for LCMs. We formulate the LCM assistance system to solve for the minimum-cost solution to merge or change lanes, which is solved efficiently using dynamic programming over the DPDM. Based on the DPDM, the LCM system recommends the required acceleration and timing to safely merge or change lanes with minimum cost. System performance has been extensively validated using real-world on-road data, including urban driving, on-ramp merges, and both dense and free-flow highway conditions.

Dynamic Probabilistic Drivability Maps for Lane Change and Merge Driver Assistance

A coherent lidar system based on the measurement of complex optical field is demonstrated for the first time. An electro-optic in-phase/quadrature (I/Q) modulator is used in the lidar transmitter to realize carrier-suppressed complex optical field modulation in which the positive and the negative optical sidebands can carry independent modulation waveforms. A fiber-optic 90° hybrid is used in the lidar receiver for coherent heterodyne detection and to recover the complex optical field. By loading a constant modulation frequency on the lower optical sideband and a wideband linear frequency chirp on the upper sideband, vector velocity and target distance can be measured independently. The wide modulation bandwidth of this lidar system also enabled unprecedented range resolution and the capability of measuring high velocity unambiguously.

https://kuscholarworks.ku.edu/bitstream/handle/1808/11546/Hui_Complex%20optical%20field%20lidar%20system.pdf;sequence=1

Complex-optical-field lidar system for range and vector velocity measurement

We present the application of optical code division multiple access (OCDMA) modulation based on optical orthogonal codes for automotive time-of-flight light detection and ranging (LiDAR) system sensors using avalanche photodiode (APD) detectors. The modulation opens the possibility to discriminate single laser transmissions. This allows the realization of additional features like enhancing the systems’ interference robustness or accelerating their scanning rate. The requirements on automotive LiDAR OCDMA modulation differs from telecommunication’s demands in several ways: The sensor must guarantee absolute laser safety; front facing devices must be able to provide reliable long range results up to a distance of 150m and beyond; and the signal is transmitted through free space. After outlining the basic functionality of the sort of sensors in consideration, the properties of the optical orthogonal codes (OOC) modulation are compared with the state of the art theoretically. The influence of OOC parameters with respect to scanning LiDAR systems is examined. The modulation technique is then demonstrated experimentally with two examples: The detection and separation of coded and traditional signals is shown using a matched filter detection algorithm. In the same way, differently coded signals can be separated from each other. Impediments of the interference suppression quality are discussed. Finally, an overview of possible applications of the proposed technique in automotive LiDAR systems is given, which are enabled by the OCDMA modulation in the first place.

A CDMA Modulation Technique for Automotive Time-of-Flight LiDAR Systems

This paper discusses the construction principles and performance of a pulsed time-of-flight (TOF) laser radar based on high-speed (FWHM $\sim$ 100 ps) and high-energy ( $\sim$ 1 nJ) optical transmitter pulses produced with a specific laser diode working in an “enhanced gain-switching” regime and based on single-photon detection in the receiver. It is shown by analysis and experiments that single-shot precision at the level of 2…3 cm is achievable. The effective measurement rate can exceed 10 kHz to a noncooperative target (20% reflectivity) at a distance of $>\ 50\ \hbox{m}$ , with an effective receiver aperture size of $2.5\ \hbox{cm}^{2} $ . The effect of background illumination is analyzed. It is shown that the gating of the SPAD detector is an effective means to avoid the blocking of the receiver in a high-level background illumination case. A brief comparison with pulsed TOF laser radars employing linear detection techniques is also made.

On Laser Ranging Based on High-Speed/Energy Laser Diode Pulses and Single-Photon Detection Techniques

Doppler laser radar can improve the precision of speed measurement by about two orders of magnitude compared with time-of-flight range finders, which obtain target speeds by range differentiation. However, in a car environment, the usage of traditional Doppler laser radar schemes is limited, because they do not satisfy the requirement of simultaneously measuring the target range together with speed with high precision. First, in this paper, we describe a new in-car laser radar system and show a new modulation scheme that enables the in-car laser radar to simultaneously measure the target range and speed with high precision. Then, we perform simulations and experiments to verify the accuracy of the proposed method. In the Appendix, a brief review of current widely used laser radar schemes is given. The limitations of these schemes to their employment in car applications are also discussed.

Amplitude-Modulated Laser Radar for Range and Speed Measurement in Car Applications

Laser radar provides better spatial resolution than millimeter-wave radar (MWR) due to the high directivity of the laser beam. However, commercial in-car laser radar approximates the target speed by a range differentiation method, which has the problems of excessive time consumption and the introduction of large errors. In this paper, a new Doppler laser radar scheme for simultaneously measuring target range and speed in automotive applications is presented. The scheme includes a new laser radar architecture, a new method to modulate the transmitted signal, and a method for calculating the range and speed from the signal returned from the target. The length of the transmitted signal is several microseconds, giving the possibility of realizing a high scan speed in automotive applications. In addition, simulations based on Simulink/Matlab were carried out to validate the proposed scheme. Finally, an experimental demonstration of simultaneous range and speed measurements was performed. The moving target for the experiment was a highly reflecting sheet attached to an electric grinder.

Demonstration of In-Car Doppler Laser Radar at 1.55 $ \mu\hbox{m}$ for Range and Speed Measurement

Laser Radar Receiver Performance Improvement by Inter Symbol Interference

In this paper, we describe an in-car laser radar schematic diagram, and propose a new modulation scheme for the in-car laser radar measuring target range and speed with high precision simultaneously Then, we perform experiments to verify the correctness of the proposed method

Doppler laser radar for measuring target range and speed with high precision

PROBLEM TO BE SOLVED: To obtain a high correlation peak by setting the cutoff frequency as appropriate to suppress a decrease in code recognition ratio when a reception signal is binarized.SOLUTION: A laser light source 26 is driven to emit a light pulse in accordance with a PN code generated in a PN code generating circuit 22, and a photodiode 32 receives a reflection light pulse having reflected on a target object to output a reception signal. The reception signal having passed through a bandpass filter 34 is shifted to a negative direction and binarized in an encoder 36, and a distance to the target object is calculated in a signal processing section 40 based on a phase difference between the reception signal and the PN code at which the correlation is the peak. The frequency set so that time until the reception signal becomes a steady state is smaller than one period of the reception signal is the lower limit of the low cutoff frequency of the bandpass filter 34, and the frequency set so that a value obtained by subtracting a pulse distortion relative amount from the peak value of the reception signal is larger than the sum of a signal level of a noise component and the threshold value of the encoder 36 is the upper limit thereof.

Xuesong Mao

Papers

Amplitude-Modulated Laser Radar for Range and Speed Measurement in Car Applications

Demonstration of In-Car Doppler Laser Radar at 1.55 $ \mu\hbox{m}$ for Range and Speed Measurement

Laser Radar Receiver Performance Improvement by Inter Symbol Interference

Doppler laser radar for measuring target range and speed with high precision

Transmission and reception signal correlation detecting device and distance measuring device