Automatic video analysis from urban surveillance cameras is a fast-emerging field based on computer vision techniques. We present here a comprehensive review of the state-of-the-art computer vision for traffic video with a critical analysis and an outlook to future research directions. This field is of increasing relevance for intelligent transport systems (ITSs). The decreasing hardware cost and, therefore, the increasing deployment of cameras have opened a wide application field for video analytics. Several monitoring objectives such as congestion, traffic rule violation, and vehicle interaction can be targeted using cameras that were typically originally installed for human operators. Systems for the detection and classification of vehicles on highways have successfully been using classical visual surveillance techniques such as background estimation and motion tracking for some time. The urban domain is more challenging with respect to traffic density, lower camera angles that lead to a high degree of occlusion, and the variety of road users. Methods from object categorization and 3-D modeling have inspired more advanced techniques to tackle these challenges. There is no commonly used data set or benchmark challenge, which makes the direct comparison of the proposed algorithms difficult. In addition, evaluation under challenging weather conditions (e.g., rain, fog, and darkness) would be desirable but is rarely performed. Future work should be directed toward robust combined detectors and classifiers for all road users, with a focus on realistic conditions during evaluation.

A Review of Computer Vision Techniques for the Analysis of Urban Traffic

This paper presents an automatic traffic surveillance system to estimate important traffic parameters from video sequences using only one camera. Different from traditional methods that can classify vehicles to only cars and noncars, the proposed method has a good ability to categorize vehicles into more specific classes by introducing a new "linearity" feature in vehicle representation. In addition, the proposed system can well tackle the problem of vehicle occlusions caused by shadows, which often lead to the failure of further vehicle counting and classification. This problem is solved by a novel line-based shadow algorithm that uses a set of lines to eliminate all unwanted shadows. The used lines are devised from the information of lane-dividing lines. Therefore, an automatic scheme to detect lane-dividing lines is also proposed. The found lane-dividing lines can also provide important information for feature normalization, which can make the vehicle size more invariant, and thus much enhance the accuracy of vehicle classification. Once all features are extracted, an optimal classifier is then designed to robustly categorize vehicles into different classes. When recognizing a vehicle, the designed classifier can collect different evidences from its trajectories and the database to make an optimal decision for vehicle classification. Since more evidences are used, more robustness of classification can be achieved. Experimental results show that the proposed method is more robust, accurate, and powerful than other traditional methods, which utilize only the vehicle size and a single frame for vehicle classification.

/pdf/automatic-traffic-surveillance-system-for-vehicle-tracking-1z4zygq2gw.pdf

Automatic traffic surveillance system for vehicle tracking and classification

In this paper, we propose a vehicle type classification method using a semisupervised convolutional neural network from vehicle frontal-view images. In order to capture rich and discriminative information of vehicles, we introduce sparse Laplacian filter learning to obtain the filters of the network with large amounts of unlabeled data. Serving as the output layer of the network, the softmax classifier is trained by multitask learning with small amounts of labeled data. For a given vehicle image, the network can provide the probability of each type to which the vehicle belongs. Unlike traditional methods by using handcrafted visual features, our method is able to automatically learn good features for the classification task. The learned features are discriminative enough to work well in complex scenes. We build the challenging BIT-Vehicle dataset, including 9850 high-resolution vehicle frontal-view images. Experimental results on our own dataset and a public dataset demonstrate the effectiveness of the proposed method.

Vehicle Type Classification Using a Semisupervised Convolutional Neural Network

Visual surveillance is an active research topic in image processing. Transit systems are actively seeking new or improved ways to use technology to deter and respond to accidents, crime, suspicious activities, terrorism, and vandalism. Human behavior-recognition algorithms can be used proactively for prevention of incidents or reactively for investigation after the fact. This paper describes the current state-of-the-art image-processing methods for automatic-behavior-recognition techniques, with focus on the surveillance of human activities in the context of transit applications. The main goal of this survey is to provide researchers in the field with a summary of progress achieved to date and to help identify areas where further research is needed. This paper provides a thorough description of the research on relevant human behavior-recognition methods for transit surveillance. Recognition methods include single person (e.g., loitering), multiple-person interactions (e.g., fighting and personal attacks), person-vehicle interactions (e.g., vehicle vandalism), and person-facility/location interactions (e.g., object left behind and trespassing). A list of relevant behavior-recognition papers is presented, including behaviors, data sets, implementation details, and results. In addition, algorithm's weaknesses, potential research directions, and contrast with commercial capabilities as advertised by manufacturers are discussed. This paper also provides a summary of literature surveys and developments of the core technologies (i.e., low-level processing techniques) used in visual surveillance systems, including motion detection, classification of moving objects, and tracking.

Understanding Transit Scenes: A Survey on Human Behavior-Recognition Algorithms

This paper presents a novel vehicle detection approach for detecting vehicles from static images using color and edges. Different from traditional methods, which use motion features to detect vehicles, this method introduces a new color transform model to find important "vehicle color" for quickly locating possible vehicle candidates. Since vehicles have various colors under different weather and lighting conditions, seldom works were proposed for the detection of vehicles using colors. The proposed new color transform model has excellent capabilities to identify vehicle pixels from background, even though the pixels are lighted under varying illuminations. After finding possible vehicle candidates, three important features, including corners, edge maps, and coefficients of wavelet transforms, are used for constructing a cascade multichannel classifier. According to this classifier, an effective scanning can be performed to verify all possible candidates quickly. The scanning process can be quickly achieved because most background pixels are eliminated in advance by the color feature. Experimental results show that the integration of global color features and local edge features is powerful in the detection of vehicles. The average accuracy rate of vehicle detection is 94.9%

Vehicle Detection Using Normalized Color and Edge Map

This paper presents algorithms for vision-based detection and classification of vehicles in monocular image sequences of traffic scenes recorded by a stationary camera. Processing is done at three levels: raw images, region level, and vehicle level. Vehicles are modeled as rectangular patches with certain dynamic behavior. The proposed method is based on the establishment of correspondences between regions and vehicles, as the vehicles move through the image sequence. Experimental results from highway scenes are provided which demonstrate the effectiveness of the method. We also briefly describe an interactive camera calibration tool that we have developed for recovering the camera parameters using features in the image selected by the user.

/pdf/detection-and-classification-of-vehicles-5c5ne232ad.pdf

Detection and classification of vehicles

This work presents a method for detecting abandoned objects in real-world conditions. The method presented here addresses the online and real time aspects of such systems, utilizes logic to differentiate between abandoned objects and stationary people, and is robust to temporary occlusion of potential abandoned objects. The capacity to not detect still people as abandoned objects is a major aspect that differentiates this work from others in the literature. Results are presented on 3 hours 36 minutes of footage over four videos representing both sparsely and densely populated real-world situations, also differentiating this work from others in the literature

Real time, online detection of abandoned objects in public areas

The ability to detect shadows is a critical feature of any intelligent transportation system (ITS). The improper handling of shadows can be the cause of erroneous conclusions in traffic analysis. As vision-based ITS applications are becoming more popular, it is important to minimize the effects of shadows. Here, we present a novel method for shadow handling in a sequence of images. Following recent work motivated by studies of the statistics of natural images, we show that this new method is a reliable detector of shadows and can be easily implemented in real-time.

Using intrinsic images for shadow handling

Neuropsychiatric disorders are highly prevalent conditions with significant individual, societal, and economic impacts. A major challenge in the diagnosis and treatment of these conditions is the lack of sensitive, reliable, objective, quantitative tools to inform diagnosis, and measure symptom severity. Currently available assays rely on self-reports and clinician observations, leading to subjective analysis. As a step toward creating quantitative assays of neuropsychiatric symptoms, we propose an immersive environment to track behaviors relevant to neuropsychiatric symptomatology and to systematically study the effect of environmental contexts on certain behaviors. Moreover, the overarching theme leads to connected tele-psychiatry which can provide effective assessment.

/pdf/the-use-of-immersive-environments-for-the-early-detection-1i2n2wmfi2.pdf

The Use of Immersive Environments for the Early Detection and Treatment of Neuropsychiatric Disorders

This report summarizes the research behind a real-time system for vehicle detection and classification in images of traffic obtained by a stationary CCD camera. The system models vehicles as rectangular bodies with appropriate dynamic behavior and processes images on three levels: raw image, blob, and vehicle. Correspondence is calculated between the processing levels as the vehicles move through the scene. This report also presents a new calibration algorithm for the camera. Implemented on a dual Pentium PC equipped with a Matrox Genesis C80 video processing board, the system performed detection and classification at a frame rate of 15 frames per second. Detection accuracy approached 95%, and classification of those detected vehicles neared 65%. The report includes an analysis of scenes from highway traffic to demonstrate this application.

Robert F. K. Martin

Papers

Detection and classification of vehicles

Real time, online detection of abandoned objects in public areas

Using intrinsic images for shadow handling

The Use of Immersive Environments for the Early Detection and Treatment of Neuropsychiatric Disorders

Algorithms for Vehicle Classification: Phase II