Automatic video analysis from urban surveillance cameras is a fast-emerging field based on computer vision techniques. We present here a comprehensive review of the state-of-the-art computer vision for traffic video with a critical analysis and an outlook to future research directions. This field is of increasing relevance for intelligent transport systems (ITSs). The decreasing hardware cost and, therefore, the increasing deployment of cameras have opened a wide application field for video analytics. Several monitoring objectives such as congestion, traffic rule violation, and vehicle interaction can be targeted using cameras that were typically originally installed for human operators. Systems for the detection and classification of vehicles on highways have successfully been using classical visual surveillance techniques such as background estimation and motion tracking for some time. The urban domain is more challenging with respect to traffic density, lower camera angles that lead to a high degree of occlusion, and the variety of road users. Methods from object categorization and 3-D modeling have inspired more advanced techniques to tackle these challenges. There is no commonly used data set or benchmark challenge, which makes the direct comparison of the proposed algorithms difficult. In addition, evaluation under challenging weather conditions (e.g., rain, fog, and darkness) would be desirable but is rarely performed. Future work should be directed toward robust combined detectors and classifiers for all road users, with a focus on realistic conditions during evaluation.

A Review of Computer Vision Techniques for the Analysis of Urban Traffic

Length-based vehicle classification data are important inputs for traffic operation, pavement design, and transportation planning. However, such data are not directly measurable by single-loop detectors, the most widely deployed type of traffic sensor in the existing roadway infrastructure. In this study a video-based vehicle detection and classification (VVDC) system was developed for truck data collection using wide-ranging available surveillance cameras. Several computer vision-based algorithms were developed or applied to extract background image from a video sequence, detect presence of vehicles, identify and remove shadows, and calculate pixel-based vehicle lengths for classification. Care was taken to handle robustly negative effects resulting from vehicle occlusions in the horizontal direction and slight camera vibrations. The pixel-represented lengths were exploited to distinguish long vehicles from short vehicles; hence the need for complicated camera calibration can be eliminated. These algorithms were implemented in the prototype VVDC system using Microsoft Visual C#. As a plug-and-play system, the VVDC system is capable of processing both digitized image streams and live video signals in real time. The system was tested at three test locations under different traffic and environmental conditions. The accuracy for vehicle detection was above 97%, and the total truck count error was lower than 9% for all three tests. This indicates that the video image processing method developed for vehicle detection and classification in this study is indeed a viable alternative for truck data collection.

/pdf/video-based-vehicle-detection-and-classification-system-for-2he3r5jx3b.pdf

Video-Based Vehicle Detection and Classification System for Real-Time Traffic Data Collection Using Uncalibrated Video Cameras

Traffic surveillance has become an important topic in intelligent transportation systems (ITSs), which is aimed at monitoring and managing traffic flow. With the progress in computer vision, video-based surveillance systems have made great advances on traffic surveillance in ITSs. However, the performance of most existing surveillance systems is susceptible to challenging complex traffic scenes (e.g., object occlusion, pose variation, and cluttered background). Moreover, existing related research is mainly on a single video sensor node, which is incapable of addressing the surveillance of traffic road networks. Accordingly, we present a review of the literature on the video-based vehicle surveillance systems in ITSs. We analyze the existing challenges in video-based surveillance systems for the vehicle and present a general architecture for video surveillance systems, i.e., the hierarchical and networked vehicle surveillance, to survey the different existing and potential techniques. Then, different methods are reviewed and discussed with respect to each module. Applications and future developments are discussed to provide future needs of ITS services.

/pdf/hierarchical-and-networked-vehicle-surveillance-in-its-a-2dq9vsx0r3.pdf

Hierarchical and Networked Vehicle Surveillance in ITS: A Survey

Multi-vehicle detection algorithm through combining Harr and HOG features

In this paper, the preliminary application of a new type of cement-based piezoelectric sensor developed recently for monitoring the traffic flows is proposed and investigated. The sensor was made by bonding between two cubes of hardened cement sand paste with a commercially available piezoelectric ceramic plate encapsulated by a new sealant developed especially for the sensor's fabrication, and its property was tested by dynamic loadings including harmonic loading, random loading, and pulse loading. Two experiments were carried out to prove the feasibility and reliability of the sensor for monitoring the traffic flows including the person flow and the vehicle flow in the field of transportation. The tested results show that the sensors can correctly respond to the pulse loadings induced by walking person or passing vehicle and the responses of sensors were approximately direct proportional to the peak values of pulse loadings. A conclusion is given that there is a good potential for the cement-based piezoelectric sensor in the engineering application for monitoring traffic flows in the field of transportation.

Application of Cement-Based Piezoelectric Sensors for Monitoring Traffic Flows

Real time classification and tracking of multiple vehicles in highways

Image annotation using multi-view non-negative matrix factorization with different number of basis vectors

Nowadays, the number of digital images has increased so that the management of this volume of data needs an efficient system for browsing, categorising and searching. Automatic image annotation is designed for assigning tags to images for more accurate retrieval. Non-negative matrix factorisation (NMF) is a traditional machine learning technique for decomposing a matrix into a set of basis and coefficients under the non-negative constraints. In this study, the authors propose a two-step algorithm for designing an automatic image annotation system that employs the NMF framework for its first step and a variant of K-nearest neighbourhood as its second step. In the first step, a new multimodal NMF algorithm is proposed to extract the latent factors which reflect the content of images. This is done by jointly factorising the visual and textual data feature matrices so that they have close representation, although not necessarily the same. In the second step, after mapping images to the latent factors space a few tags are predicted for the new images based on a weighted average of similar data. They evaluated the performance of the proposed method and compared it to the state-of-the-art literature. Comparison results demonstrate the effectiveness and potential of the proposed method in image annotation applications.

Automatic image annotation by a loosely joint non-negative matrix factorisation

A multi-criteria decision based on adaptive routing algorithms with discrete operators for on-chip networks

In automatic image annotation (AIA) different features describe images from different aspects or views. Part of information embedded in some views is common for all views, while other parts are individual and specific. In this paper, we present the Mvg-NMF approach, a multi-view-group non-negative matrix factorization (NMF) method for an AIA system which considers both common and individual factors. The NMF framework discovers a latent space by decomposing data into a set of non-negative basis vectors and coefficients. The views divided into homogeneous groups and latent spaces are extracted for each group. After mapping the test images into these spaces, a unified distance matrix is computed from the distance between images in all spaces. Then a search-based method is used to propagate tags from the nearest neighbors to test images. The evaluation on three datasets commonly used for image annotation showed that the Mvg-NMF is highly competitive with the recent state-of-the-art works.

Roya Rad

Papers

Real time classification and tracking of multiple vehicles in highways

Image annotation using multi-view non-negative matrix factorization with different number of basis vectors

Automatic image annotation by a loosely joint non-negative matrix factorisation

A multi-criteria decision based on adaptive routing algorithms with discrete operators for on-chip networks

A multi-view-group non-negative matrix factorization approach for automatic image annotation