For the last two decades, intelligent transportation systems (ITS) have emerged as an efficient way of improving the performance of transportation systems, enhancing travel security, and providing more choices to travelers. A significant change in ITS in recent years is that much more data are collected from a variety of sources and can be processed into various forms for different stakeholders. The availability of a large amount of data can potentially lead to a revolution in ITS development, changing an ITS from a conventional technology-driven system into a more powerful multifunctional data-driven intelligent transportation system (D2ITS) : a system that is vision, multisource, and learning algorithm driven to optimize its performance. Furthermore, D2ITS is trending to become a privacy-aware people-centric more intelligent system. In this paper, we provide a survey on the development of D2ITS, discussing the functionality of its key components and some deployment issues associated with D2ITS Future research directions for the development of D2ITS is also presented.

Data-Driven Intelligent Transportation Systems: A Survey

This paper provides a review of the literature in on-road vision-based vehicle detection, tracking, and behavior understanding. Over the past decade, vision-based surround perception has progressed from its infancy into maturity. We provide a survey of recent works in the literature, placing vision-based vehicle detection in the context of sensor-based on-road surround analysis. We detail advances in vehicle detection, discussing monocular, stereo vision, and active sensor-vision fusion for on-road vehicle detection. We discuss vision-based vehicle tracking in the monocular and stereo-vision domains, analyzing filtering, estimation, and dynamical models. We discuss the nascent branch of intelligent vehicles research concerned with utilizing spatiotemporal measurements, trajectories, and various features to characterize on-road behavior. We provide a discussion on the state of the art, detail common performance metrics and benchmarks, and provide perspective on future research directions in the field.

https://cvrr.ucsd.edu/publications/2013/SayananTrivedi_IEEETITS2013_VehicleSurvey.pdf

Looking at Vehicles on the Road: A Survey of Vision-Based Vehicle Detection, Tracking, and Behavior Analysis

Driving style can characteristically be divided into two categories: “typical” (non-aggressive) and aggressive. Understanding and recognizing driving events that fall into these categories can aid in vehicle safety systems. Potentially-aggressive driving behavior is currently a leading cause of traffic fatalities in the United States. More often than not, drivers are unaware that they commit potentially-aggressive actions daily. To increase awareness and promote driver safety, we are proposing a novel system that uses Dynamic Time Warping (DTW) and smartphone based sensor-fusion (accelerometer, gyroscope, magnetometer, GPS, video) to detect, recognize and record these actions without external processing. Our system differs from past driving pattern recognition research by fusing related inter-axial data from multiple sensors into a single classifier. It also utilizes Euler representation of device attitude (also based on fused data) to aid in classification. All processing is done completely on the smartphone.

Driving style recognition using a smartphone as a sensor platform

Forecasting the motion of surrounding vehicles is a critical ability for an autonomous vehicle deployed in complex traffic. Motion of all vehicles in a scene is governed by the traffic context, i.e., the motion and relative spatial configuration of neighboring vehicles. In this paper we propose an LSTM encoder-decoder model that uses convolutional social pooling as an improvement to social pooling layers for robustly learning interdependencies in vehicle motion. Additionally, our model outputs a multi-modal predictive distribution over future trajectories based on maneuver classes. We evaluate our model using the publicly available NGSIM US-101 and I-80 datasets. Our results show improvement over the state of the art in terms of RMS values of prediction error and negative log-likelihoods of true future trajectories under the model's predictive distribution. We also present a qualitative analysis of the model's predicted distributions for various traffic scenarios.

/pdf/convolutional-social-pooling-for-vehicle-trajectory-4fzlp3b04v.pdf

Convolutional Social Pooling for Vehicle Trajectory Prediction

Perception system design is a vital step in the development of an autonomous vehicle (AV). With the vast selection of available off-the-shelf schemes and seemingly endless options of sensor systems implemented in research and commercial vehicles, it can be difficult to identify the optimal system for one’s AV application. This article presents a comprehensive review of the state-of-the-art AV perception technology available today. It provides up-to-date information about the advantages, disadvantages, limits, and ideal applications of specific AV sensors; the most prevalent sensors in current research and commercial AVs; autonomous features currently on the market; and localization and mapping methods currently implemented in AV research. This information is useful for newcomers to the AV field to gain a greater understanding of the current AV solution landscape and to guide experienced researchers towards research areas requiring further development. Furthermore, this paper highlights future research areas and draws conclusions about the most effective methods for AV perception and its effect on localization and mapping. Topics discussed in the Perception and Automotive Sensors section focus on the sensors themselves, whereas topics discussed in the Localization and Mapping section focus on how the vehicle perceives where it is on the road, providing context for the use of the automotive sensors. By improving on current state-of-the-art perception systems, AVs will become more robust, reliable, safe, and accessible, ultimately providing greater efficiency, mobility, and safety benefits to the public.

Autonomous vehicle perception: The technology of today and tomorrow

This paper introduces a general active-learning framework for robust on-road vehicle recognition and tracking. This framework takes a novel active-learning approach to building vehicle-recognition and tracking systems. A passively trained recognition system is built using conventional supervised learning. Using the query and archiving interface for active learning (QUAIL), the passively trained vehicle-recognition system is evaluated on an independent real-world data set, and informative samples are queried and archived to perform selective sampling. A second round of learning is then performed to build an active-learning-based vehicle recognizer. Particle filter tracking is integrated to build a complete multiple-vehicle tracking system. The active-learning-based vehicle-recognition and tracking (ALVeRT) system has been thoroughly evaluated on static images and roadway video data captured in a variety of traffic, illumination, and weather conditions. Experimental results show that this framework yields a robust efficient on-board vehicle recognition and tracking system with high precision, high recall, and good localization.

https://cvrr.ucsd.edu/publications/2010/sivaraman_ITS10.pdf

A General Active-Learning Framework for On-Road Vehicle Recognition and Tracking

In this paper, we introduce a synergistic approach to integrated lane and vehicle tracking for driver assistance. The approach presented in this paper results in a final system that improves on the performance of both lane tracking and vehicle tracking modules. Further, the presented approach introduces a novel approach to localizing and tracking other vehicles on the road with respect to lane position, which provides information on higher contextual relevance that neither the lane tracker nor vehicle tracker can provide by itself. Improvements in lane tracking and vehicle tracking have been extensively quantified. Integrated system performance has been validated on real-world highway data. Without specific hardware and software optimizations, the fully implemented system runs at near-real-time speeds of 11 frames per second.

https://cvrr.ucsd.edu/publications/2013/SayananTrivedi_IEEETITS-june2013.pdf

Integrated Lane and Vehicle Detection, Localization, and Tracking: A Synergistic Approach

In recent years, active learning has emerged as a powerful tool in building robust systems for object detection using computer vision. Indeed, active learning approaches to on-road vehicle detection have achieved impressive results. While active learning approaches for object detection have been explored and presented in the literature, few studies have been performed to comparatively assess costs and merits. In this study, we provide a cost-sensitive analysis of three popular active learning methods for on-road vehicle detection. The generality of active learning findings is demonstrated via learning experiments performed with detectors based on histogram of oriented gradient features and SVM classification (HOG---SVM), and Haar-like features and Adaboost classification (Haar---Adaboost). Experimental evaluation has been performed on static images and real-world on-road vehicle datasets. Learning approaches are assessed in terms of the time spent annotating, data required, recall, and precision.

http://cvrr.ucsd.edu/publications/2011/sivaraman_mva_2011.pdf

Active learning for on-road vehicle detection: a comparative study

This paper details the research, development, and demonstrations of real-world systems intended to assist the driver in urban environments, as part of the Urban Intelligent Assist (UIA) research initiative. A 3-year collaboration between Audi AG, Volkswagen Group of America Electronics Research Laboratory, and UC San Diego, the driver assistance portion of the UIA project focuses on two main use cases of vital importance in urban driving. The first, Driver Attention Guard, applies novel computer vision and machine learning research for accurately tracking the driver's head position and rotation using an array of cameras. The system then infers the driver's focus of attention, alerting the driver and engaging safety systems in case of extended driver inattention. The second application, Merge and Lane Change Assist, applies a novel probabilistic compact representation of the on-road environment, fusing data from a variety of sensor modalities. The system then computes safe and low-cost merge and lane-change maneuver recommendations. It communicates desired speeds to the driver via Head-up Display, when the driver touches the blinker, indicating his desired lane. The fully-implemented systems, complete with HMI, were demonstrated to the public and press in San Francisco in January of 2014.

https://cvrr.ucsd.edu/publications/2014/TawariSivaramanTrivedi_IV2014.pdf

Looking-in and looking-out vision for Urban Intelligent Assistance: Estimation of driver attentive state and dynamic surround for safe merging and braking

Sayanan Sivaraman

Papers

Looking at Vehicles on the Road: A Survey of Vision-Based Vehicle Detection, Tracking, and Behavior Analysis

A General Active-Learning Framework for On-Road Vehicle Recognition and Tracking

Integrated Lane and Vehicle Detection, Localization, and Tracking: A Synergistic Approach

Active learning for on-road vehicle detection: a comparative study

Looking-in and looking-out vision for Urban Intelligent Assistance: Estimation of driver attentive state and dynamic surround for safe merging and braking