Visual understanding of complex urban street scenes is an enabling factor for a wide range of applications. Object detection has benefited enormously from large-scale datasets, especially in the context of deep learning. For semantic urban scene understanding, however, no current dataset adequately captures the complexity of real-world urban scenes. To address this, we introduce Cityscapes, a benchmark suite and large-scale dataset to train and test approaches for pixel-level and instance-level semantic labeling. Cityscapes is comprised of a large, diverse set of stereo video sequences recorded in streets from 50 different cities. 5000 of these images have high quality pixel-level annotations, 20 000 additional images have coarse annotations to enable methods that leverage large volumes of weakly-labeled data. Crucially, our effort exceeds previous attempts in terms of dataset size, annotation richness, scene variability, and complexity. Our accompanying empirical study provides an in-depth analysis of the dataset characteristics, as well as a performance evaluation of several state-of-the-art approaches based on our benchmark.

https://www.cityscapes-dataset.com/wordpress/wp-content/papercite-data/pdf/cordts2016cityscapes.pdf

The Cityscapes Dataset for Semantic Urban Scene Understanding

An overview of the self-organizing map algorithm, on which the papers in this issue are based, is presented in this article.

The Self-Organizing Map

There is a large variety of trackers, which have been proposed in the literature during the last two decades with some mixed success. Object tracking in realistic scenarios is a difficult problem, therefore, it remains a most active area of research in computer vision. A good tracker should perform well in a large number of videos involving illumination changes, occlusion, clutter, camera motion, low contrast, specularities, and at least six more aspects. However, the performance of proposed trackers have been evaluated typically on less than ten videos, or on the special purpose datasets. In this paper, we aim to evaluate trackers systematically and experimentally on 315 video fragments covering above aspects. We selected a set of nineteen trackers to include a wide variety of algorithms often cited in literature, supplemented with trackers appearing in 2010 and 2011 for which the code was publicly available. We demonstrate that trackers can be evaluated objectively by survival curves, Kaplan Meier statistics, and Grubs testing. We find that in the evaluation practice the F-score is as effective as the object tracking accuracy (OTA) score. The analysis under a large variety of circumstances provides objective insight into the strengths and weaknesses of trackers.

/pdf/visual-tracking-an-experimental-survey-50w2t40998.pdf

Visual Tracking: An Experimental Survey

Intelligent vehicles have increased their capabilities for highly and, even fully, automated driving under controlled environments. Scene information is received using onboard sensors and communication network systems, i.e., infrastructure and other vehicles. Considering the available information, different motion planning and control techniques have been implemented to autonomously driving on complex environments. The main goal is focused on executing strategies to improve safety, comfort, and energy optimization. However, research challenges such as navigation in urban dynamic environments with obstacle avoidance capabilities, i.e., vulnerable road users (VRU) and vehicles, and cooperative maneuvers among automated and semi-automated vehicles still need further efforts for a real environment implementation. This paper presents a review of motion planning techniques implemented in the intelligent vehicles literature. A description of the technique used by research teams, their contributions in motion planning, and a comparison among these techniques is also presented. Relevant works in the overtaking and obstacle avoidance maneuvers are presented, allowing the understanding of the gaps and challenges to be addressed in the next years. Finally, an overview of future research direction and applications is given.

A Review of Motion Planning Techniques for Automated Vehicles

This paper provides a review of the literature in on-road vision-based vehicle detection, tracking, and behavior understanding. Over the past decade, vision-based surround perception has progressed from its infancy into maturity. We provide a survey of recent works in the literature, placing vision-based vehicle detection in the context of sensor-based on-road surround analysis. We detail advances in vehicle detection, discussing monocular, stereo vision, and active sensor-vision fusion for on-road vehicle detection. We discuss vision-based vehicle tracking in the monocular and stereo-vision domains, analyzing filtering, estimation, and dynamical models. We discuss the nascent branch of intelligent vehicles research concerned with utilizing spatiotemporal measurements, trajectories, and various features to characterize on-road behavior. We provide a discussion on the state of the art, detail common performance metrics and benchmarks, and provide perspective on future research directions in the field.

https://cvrr.ucsd.edu/publications/2013/SayananTrivedi_IEEETITS2013_VehicleSurvey.pdf

Looking at Vehicles on the Road: A Survey of Vision-Based Vehicle Detection, Tracking, and Behavior Analysis

In this paper, an efficient real-time autonomous driving motion planner with trajectory optimization is proposed. The planner first discretizes the plan space and searches for the best trajectory based on a set of cost functions. Then an iterative optimization is applied to both the path and speed of the resultant trajectory. The post-optimization is of low computational complexity and is able to converge to a higher-quality solution within a few iterations. Compared with the planner without optimization, this framework can reduce the planning time by 52% and improve the trajectory quality. The proposed motion planner is implemented and tested both in simulation and on a real autonomous vehicle in three different scenarios. Experiments show that the planner outputs high-quality trajectories and performs intelligent driving behaviors.

/pdf/a-real-time-motion-planner-with-trajectory-optimization-for-54gyy4v0du.pdf

A real-time motion planner with trajectory optimization for autonomous vehicles

We propose a novel system for tracking pedestrians in a wide and open area, such as a shopping mall and exhibition hall, using a number of single-row laser-range scanners (LD-A), which have a profiling rate of 10 Hz and a scanning angle of 270/spl deg/. LD-As are set directly on the floor doing horizontal scanning at an elevation of about 20 cm above the ground, so that horizontal cross sections of the surroundings, containing moving feet of pedestrians as well as still objects, are obtained in a rectangular coordinate system of real dimension. The data of moving feet are extracted through background subtraction by the client computers that control each LD-A, and sent to a server computer, where they are spatially and temporally integrated into a global coordinate system. A simplified pedestrian's walking model based on the typical appearance of moving feet is defined and a tracking method utilizing Kalman filter is developed to track pedestrian's trajectories. The system is evaluated through both real experiment and computer simulation. A real experiment is conducted in an exhibition hall, where three LD-As are used covering an area of about 60/spl times/60 m/sup 2/. Changes in visitors' flow during the whole exhibition day are analyzed, where in the peak hour, about 100 trajectories are extracted simultaneously. On the other hand, a computer simulation is conducted to quantitatively examine system performance with respect to different crowd density.

/pdf/a-novel-system-for-tracking-pedestrians-using-multiple-36tj91nd8y.pdf

A novel system for tracking pedestrians using multiple single-row laser-range scanners

In this paper, we present an interactive visual analytics system, Triple Perspective Visual Trajectory Analytics (TripVista), for exploring and analyzing complex traffic trajectory data. The users are equipped with a carefully designed interface to inspect data interactively from three perspectives (spatial, temporal and multi-dimensional views). While most previous works, in both visualization and transportation research, focused on the macro aspects of traffic flows, we develop visualization methods to investigate and analyze microscopic traffic patterns and abnormal behaviors. In the spatial view of our system, traffic trajectories with various presentation styles are directly interactive with user brushing, together with convenient pattern exploration and selection through ring-style sliders. Improved ThemeRiver, embedded with glyphs indicating directional information, and multiple scatterplots with time as horizontal axes illustrate temporal information of the traffic flows. Our system also harnesses the power of parallel coordinates to visualize the multi-dimensional aspects of the traffic trajectory data. The above three view components are linked closely and interactively to provide access to multiple perspectives for users. Experiments show that our system is capable of effectively finding both regular and abnormal traffic flow patterns.

/pdf/tripvista-triple-perspective-visual-trajectory-analytics-and-1pnpx6x0y0.pdf

TripVista: Triple Perspective Visual Trajectory Analytics and its application on microscopic traffic data at a road intersection

Tracking generic human motion is highly challenging due to its high-dimensional state space and the various motion types involved. In order to deal with these challenges, a fusion formulation which integrates low- and high-dimensional tracking approaches into one framework is proposed. The low-dimensional approach successfully overcomes the high-dimensional problem of tracking the motions with available training data by learning motion models, but it only works with specific motion types. On the other hand, although the high-dimensional approach may recover the motions without learned models by sampling directly in the pose space, it lacks robustness and efficiency. Within the framework, the two parallel approaches, low- and high-dimensional, are fused via a probabilistic approach at each time step. This probabilistic fusion approach ensures that the overall performance of the system is improved by concentrating on the respective advantages of the two approaches and resolving their weak points. The experimental results, after qualitative and quantitative comparisons, demonstrate the effectiveness of the proposed approach in tracking generic human motion.

http://www.bmva.org/bmvc/2011/proceedings/paper57/abstract.pdf

Tracking Generic Human Motion via Fusion of Low- and High-Dimensional Approaches

. In this paper, a novel method is presented for generating a textured CAD model of an outdoor urban environment using a vehicle-borne sensor system. In data measurement, three single-row laser range scanners and six line cameras are mounted on a measurement vehicle, which has been equipped with a GPS/INS/Odometer-based navigation system. Laser range and line images are measured as the vehicle moves forward. They are synchronized with the navigation system so they can be geo-referenced to a world coordinate system. Generation of the CAD model is conducted in two steps. A geometric model is first generated using the geo-referenced laser range data, where urban features, such as buildings, ground surfaces, and trees are extracted in a hierarchical way. Different urban features are represented using different geometric primitives, such as a planar face, a triangulated irregular network (TIN), and a triangle. The texture of the urban features is generated by projecting and resampling line images onto the geometric model. An outdoor experiment is conducted, and a textured CAD model of a real urban environment is reconstructed in a full automatic mode.

Huijing Zhao

Papers

A real-time motion planner with trajectory optimization for autonomous vehicles

A novel system for tracking pedestrians using multiple single-row laser-range scanners

TripVista: Triple Perspective Visual Trajectory Analytics and its application on microscopic traffic data at a road intersection

Tracking Generic Human Motion via Fusion of Low- and High-Dimensional Approaches

Reconstructing a textured CAD model of an urban environment using vehicle-borne laser range scanners and line cameras