This paper provides a review of the literature in on-road vision-based vehicle detection, tracking, and behavior understanding. Over the past decade, vision-based surround perception has progressed from its infancy into maturity. We provide a survey of recent works in the literature, placing vision-based vehicle detection in the context of sensor-based on-road surround analysis. We detail advances in vehicle detection, discussing monocular, stereo vision, and active sensor-vision fusion for on-road vehicle detection. We discuss vision-based vehicle tracking in the monocular and stereo-vision domains, analyzing filtering, estimation, and dynamical models. We discuss the nascent branch of intelligent vehicles research concerned with utilizing spatiotemporal measurements, trajectories, and various features to characterize on-road behavior. We provide a discussion on the state of the art, detail common performance metrics and benchmarks, and provide perspective on future research directions in the field.

https://cvrr.ucsd.edu/publications/2013/SayananTrivedi_IEEETITS2013_VehicleSurvey.pdf

Looking at Vehicles on the Road: A Survey of Vision-Based Vehicle Detection, Tracking, and Behavior Analysis

A rapid learning algorithm for vehicle classification

From the Publisher:
Real-Time Systems is both a valuable reference for professionals and an advanced text for Computer Science and Computer Engineering students. 

Real world real-time applications based on research and practice

State-of-the-art algorithms and methods for validation

Methods for end-to-end scheduling and resource management

More than 100 illustrations to enhance understanding

Comprehensive treatment of the technology known as RMA (rate-monotonic analysis) methods

A supplemental Companion Website www.prenhall.com/liu

http://user.it.uu.se/~yi/courses/rts/dvp-rts-08/notes/Course-INFO.pdf

Real-Time Systems

Event analysis in videos is a critical task in many applications. Activity recognition that aims to recognize actions from video and in particular abnormal event recognition in surveillance video has received significant attention from the research community. In this special issue, we focus on event analysis in broad problem domains. Event recognition in specific domains, such as highlight detection in sports videos, has attracted much interest in the past decade. Recently, due to the emergence of online video search, the research community has become interested in event content analysis for both broadcast and user-generated videos. For news videos, Large-Scale Concept Ontology for Multimedia (LSCOM) has defined 56 event/activity concepts, covering a broad range of events such as airplane flying, car crash, riot, people marching, and so on. Researchers have also started to investigate event recognition from other video sources, such as education videos and medical videos. For these applications, we have witnessed the effectiveness of using both static and temporal information.

IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY Editor-in-Chief

Over the past decade, vision-based vehicle detection techniques for road safety improvement have gained an increasing amount of attention. Unfortunately, the techniques suffer from robustness due to huge variability in vehicle shape (particularly for motorcycles), cluttered environment, various illumination conditions, and driving behavior. In this paper, we provide a comprehensive survey in a systematic approach about the state-of-the-art on-road vision-based vehicle detection and tracking systems for collision avoidance systems (CASs). This paper is structured based on a vehicle detection processes starting from sensor selection to vehicle detection and tracking. Techniques in each process/step are reviewed and analyzed individually. Two main contributions in this paper are the following: survey on motorcycle detection techniques and the sensor comparison in terms of cost and range parameters. Finally, the survey provides an optimal choice with a low cost and reliable CAS design in vehicle industries.

Vehicle Detection Techniques for Collision Avoidance Systems: A Review

A monocular vision based rear vehicle detection and tracking system is presented for Lane Change Assist (LCA), which does not need road boundary and lane information. Our algorithm extracts regions of interest (ROI) using the shadow underneath a vehicle, and accurately localizes vehicle regions in ROI by vehicle features such as symmetry, edge and shadow underneath vehicles. The algorithm realizes vehicle verification by combining knowledge-based and learning-based methods. During vehicle tracking, templates are dynamically created on-line, tracking window is adaptively adjusted with motion estimation, and confidence is determined for tracked vehicle. The algorithm was tested under various traffic scenes at different daytime, the result illustrated good performance.

Rear Vehicle Detection and Tracking for Lane Change Assist

Detection and tracking of lane marking is essential for driving safety and intelligent vehicle. In this paper, an algorithm is presented which allows detection and tracking of multiple lane markings. Edge points cue is used to detect the lane marking and a road orientation estimation method is used to delete the edge lines which are impossible attribute to lane markings. In order to select the candidate lane marking, a confidence measures method is proposed. Then a finite-state machine decides whether or not a lane marking is really detected by fusion multi-frame detection results. Specifically, a particle filter is used to predict the future values of the lane marking model parameters, based on past observations. With particle filtering and confidence measures method, lane markings on various road scenes are detected and tracked. Experimental in different conditions, including illumination, weather and road, demonstrates its effectiveness and robustness. The algorithm runs in real-time at rates of about 30 Hz.

Vision-Based Real-Time Lane Marking Detection and Tracking

A real-time monocular vision based rear vehicle and motorcycle detection and tracking approach is presented for Lane Change Assistant (LCA). To achieve robustness and accuracy this work detects and tracks multiple vehicles and motorcycles on road by combining multiple cues. To achieve real-time multi-resolution technology is used to reduce computing complexity, and all algorithms have been implemented on an IMAP (Integrated Memory Array Processor) parallel vision board. The test results under various traffic scenes illustrated accuracy, robustness and real-time of this work.

Real-time on-road vehicle and motorcycle detection using a single camera

Vision-based pedestrian detection remains a challenging task, so far. The detection performance often suffers from the various appearances of pedestrians, the illumination changes, and the possible partial occlusions. Aiming at resolving these challenges, in this paper, a new linear kernel function is proposed to effectively combine two heterogeneous features, i.e., histogram of oriented gradient and local binary pattern, which enhances the pedestrian description ability to illumination conditions and cluttered background. Then, a novel multi-view–pose part ensemble (MVPPE) detector is proposed, in order to better handle pedestrian variability, views, and partial occlusions. Experimental results in public data sets demonstrate that the proposed feature combination method significantly improves the description capabilities of pedestrian features. Compared with the existing multipart ensemble approaches, the proposed MVPPE detector boosts higher detection accuracy.

A Pedestrian-Detection Method Based on Heterogeneous Features and Ensemble of Multi-View–Pose Parts

There are many timing unreliable computing components in modern computer systems, which are typically forbidden in hard real-time systems due to the timing uncertainty. In this paper, we propose a computation offloading mechanism to utilise these timing unreliable components in a hard real-time system, by providing local compensations. The key of the mechanism is to decide (1) how the unreliable components are utilized and (2) how to set the worst-case estimated response time. The local compensation has to start when the unreliable components do not deliver the results in the estimated response time. We propose a scheduling algorithm and its schedulability test to analyze the feasibility of the compensation mechanism. To validate the proposed mechanism, we perform a case study based on image-processing applications in a robot system and simulations. By adopting the timing unreliable components, the system can handle higher-quality images and with better performance.

Wei Liu

Papers

Rear Vehicle Detection and Tracking for Lane Change Assist

Vision-Based Real-Time Lane Marking Detection and Tracking

Real-time on-road vehicle and motorcycle detection using a single camera

A Pedestrian-Detection Method Based on Heterogeneous Features and Ensemble of Multi-View–Pose Parts

Computation Offloading by Using Timing Unreliable Components in Real-Time Systems