There is, I think, something ethereal about i —the square root of minus one. I remember first hearing about it at school. It seemed an odd beast at that time—an intruder hovering on the edge of reality.

Usually familiarity dulls this sense of the bizarre, but in the case of i it was the reverse: over the years the sense of its surreal nature intensified. It seemed that it was impossible to write mathematics that described the real world in …

I and i

Today, visual recognition systems are still rarely employed in robotics applications. Perhaps one of the main reasons for this is the lack of demanding benchmarks that mimic such scenarios. In this paper, we take advantage of our autonomous driving platform to develop novel challenging benchmarks for the tasks of stereo, optical flow, visual odometry/SLAM and 3D object detection. Our recording platform is equipped with four high resolution video cameras, a Velodyne laser scanner and a state-of-the-art localization system. Our benchmarks comprise 389 stereo and optical flow image pairs, stereo visual odometry sequences of 39.2 km length, and more than 200k 3D object annotations captured in cluttered scenarios (up to 15 cars and 30 pedestrians are visible per image). Results from state-of-the-art algorithms reveal that methods ranking high on established datasets such as Middlebury perform below average when being moved outside the laboratory to the real world. Our goal is to reduce this bias by providing challenging benchmarks with novel difficulties to the computer vision community. Our benchmarks are available online at: www.cvlibs.net/datasets/kitti

/pdf/are-we-ready-for-autonomous-driving-the-kitti-vision-zygefho352.pdf

Are we ready for autonomous driving? The KITTI vision benchmark suite

Probability Distributions.- Linear Models for Regression.- Linear Models for Classification.- Neural Networks.- Kernel Methods.- Sparse Kernel Machines.- Graphical Models.- Mixture Models and EM.- Approximate Inference.- Sampling Methods.- Continuous Latent Variables.- Sequential Data.- Combining Models.

Pattern Recognition and Machine Learning

Visual understanding of complex urban street scenes is an enabling factor for a wide range of applications. Object detection has benefited enormously from large-scale datasets, especially in the context of deep learning. For semantic urban scene understanding, however, no current dataset adequately captures the complexity of real-world urban scenes. To address this, we introduce Cityscapes, a benchmark suite and large-scale dataset to train and test approaches for pixel-level and instance-level semantic labeling. Cityscapes is comprised of a large, diverse set of stereo video sequences recorded in streets from 50 different cities. 5000 of these images have high quality pixel-level annotations, 20 000 additional images have coarse annotations to enable methods that leverage large volumes of weakly-labeled data. Crucially, our effort exceeds previous attempts in terms of dataset size, annotation richness, scene variability, and complexity. Our accompanying empirical study provides an in-depth analysis of the dataset characteristics, as well as a performance evaluation of several state-of-the-art approaches based on our benchmark.

https://www.cityscapes-dataset.com/wordpress/wp-content/papercite-data/pdf/cordts2016cityscapes.pdf

The Cityscapes Dataset for Semantic Urban Scene Understanding

We present a novel dataset captured from a VW station wagon for use in mobile robotics and autonomous driving research. In total, we recorded 6 hours of traffic scenarios at 10-100 Hz using a variety of sensor modalities such as high-resolution color and grayscale stereo cameras, a Velodyne 3D laser scanner and a high-precision GPS/IMU inertial navigation system. The scenarios are diverse, capturing real-world traffic situations, and range from freeways over rural areas to inner-city scenes with many static and dynamic objects. Our data is calibrated, synchronized and timestamped, and we provide the rectified and raw image sequences. Our dataset also contains object labels in the form of 3D tracklets, and we provide online benchmarks for stereo, optical flow, object detection and other tasks. This paper describes our recording platform, the data format and the utilities that we provide.

/pdf/vision-meets-robotics-the-kitti-dataset-2h3kd0lpgn.pdf

Vision meets robotics: The KITTI dataset

This paper presents a novel semantic categorization method for 3D point cloud data using supervised, multiclass Gaussian Process (GP) classification. In contrast to other approaches, and particularly Support Vector Machines, which probably are the most used method for this task to date, GPs have the major advantage of providing informative uncertainty estimates about the resulting class labels. As we show in experiments, these uncertainty estimates can either be used to improve the classification by neglecting uncertain class labels or - more importantly - they can serve as an indication of the under-representation of certain classes in the training data. This means that GP classifiers are much better suited in a lifelong learning framework, where not all classes are represented initially, but instead new training data arrives during the operation of the robot.

/pdf/semantic-categorization-of-outdoor-scenes-with-uncertainty-1kfwa8c7hx.pdf

Semantic categorization of outdoor scenes with uncertainty estimates using multi-class gaussian process classification

A method of detecting the structural elements within a scene sensed by at least one sensor within a locale, the method comprising: a) capturing data from the sensor, which data provides a first representation of the sensed scene at the current time; b) generating a second representation of the sensed scene where the second representation is generated from a prior model of the locale; and c) comparing the first and second representations with one another to determine which parts of the first representation represent structural elements of the locale.

A method of detecting structural parts of a scene

This paper presents a framework for path planning over probabilistic costmaps of outdoor terrain that is compatible with fast grid-based planners such as A $^*$ and D $^*$ . We begin with an exemplar of how probabilistic costmaps may be constructed and then show how the a priori availability of such maps lends itself to the precomputation of exact probabilistic heuristics. In turn, the probabilistic nature of these heuristics allow the user to employ a bounded speed–accuracy tradeoff that characterizes the risk of paths returned not being of optimal shortest-path length. Results are shown which demonstrate that the method is able to closely approximate a probability distribution over the underlying exact distance and that efficiency increases on the order of 90% in terms of nodes expanded, and 60% in terms of search time over Euclidean distance heuristics, can be achieved.

Risky Planning on Probabilistic Costmaps for Path Planning in Outdoor Environments

This paper is about road vehicle localisation based on vision using synthesised local orthographic imagery. We exploit state of the art stereo visual odometry (VO) on our survey vehicle to generate high precision synthetic orthographic images of the road surface as would be seen from overhead. The fidelity and detail of these images far exceeds that of aerial photographs. When undertaking subsequent passes of the same route, the vehicle is localised against the survey vehicle's trajectory by maximising the mutual information between the synthetic orthographic images and live image streams. Thus we explicitly leverage the gross appearance of the workspace rather than a discrete set of point features. We test our technique on data gathered from a road vehicle and show that centimeter-level precision is possible without the complexity and instability of contemporary feature based techniques.

/pdf/generation-and-exploitation-of-local-orthographic-imagery-4wcgt8gntr.pdf

Generation and exploitation of local orthographic imagery for road vehicle localisation

Estimating motion from images is a well-studied problem in computer vision and robotics. Previous work has developed techniques to estimate the motion of a moving camera in a largely static environment (e.g., visual odometry) and to segment or track motions in a dynamic scene using known camera motions (e.g., multiple object tracking). 
It is more challenging to estimate the unknown motion of the camera and the dynamic scene simultaneously. Most previous work requires a priori object models (e.g., tracking-by-detection), motion constraints (e.g., planar motion), or fails to estimate the full SE(3) motions of the scene (e.g., scene flow). While these approaches work well in specific application domains, they are not generalizable to unconstrained motions. 
This paper extends the traditional visual odometry (VO) pipeline to estimate the full SE(3) motion of both a stereo/RGB-D camera and the dynamic scene. This multimotion visual odometry (MVO) pipeline requires no a priori knowledge of the environment or the dynamic objects. Its performance is evaluated on a real-world dynamic dataset with ground truth for all motions from a motion capture system.

Paul Newman

Papers

Semantic categorization of outdoor scenes with uncertainty estimates using multi-class gaussian process classification

A method of detecting structural parts of a scene

Risky Planning on Probabilistic Costmaps for Path Planning in Outdoor Environments

Generation and exploitation of local orthographic imagery for road vehicle localisation

Multimotion Visual Odometry (MVO): Simultaneous Estimation of Camera and Third-Party Motions.