There is, I think, something ethereal about i —the square root of minus one. I remember first hearing about it at school. It seemed an odd beast at that time—an intruder hovering on the edge of reality.

Usually familiarity dulls this sense of the bizarre, but in the case of i it was the reverse: over the years the sense of its surreal nature intensified. It seemed that it was impossible to write mathematics that described the real world in …

I and i

Today, visual recognition systems are still rarely employed in robotics applications. Perhaps one of the main reasons for this is the lack of demanding benchmarks that mimic such scenarios. In this paper, we take advantage of our autonomous driving platform to develop novel challenging benchmarks for the tasks of stereo, optical flow, visual odometry/SLAM and 3D object detection. Our recording platform is equipped with four high resolution video cameras, a Velodyne laser scanner and a state-of-the-art localization system. Our benchmarks comprise 389 stereo and optical flow image pairs, stereo visual odometry sequences of 39.2 km length, and more than 200k 3D object annotations captured in cluttered scenarios (up to 15 cars and 30 pedestrians are visible per image). Results from state-of-the-art algorithms reveal that methods ranking high on established datasets such as Middlebury perform below average when being moved outside the laboratory to the real world. Our goal is to reduce this bias by providing challenging benchmarks with novel difficulties to the computer vision community. Our benchmarks are available online at: www.cvlibs.net/datasets/kitti

/pdf/are-we-ready-for-autonomous-driving-the-kitti-vision-zygefho352.pdf

Are we ready for autonomous driving? The KITTI vision benchmark suite

Probability Distributions.- Linear Models for Regression.- Linear Models for Classification.- Neural Networks.- Kernel Methods.- Sparse Kernel Machines.- Graphical Models.- Mixture Models and EM.- Approximate Inference.- Sampling Methods.- Continuous Latent Variables.- Sequential Data.- Combining Models.

Pattern Recognition and Machine Learning

Visual understanding of complex urban street scenes is an enabling factor for a wide range of applications. Object detection has benefited enormously from large-scale datasets, especially in the context of deep learning. For semantic urban scene understanding, however, no current dataset adequately captures the complexity of real-world urban scenes. To address this, we introduce Cityscapes, a benchmark suite and large-scale dataset to train and test approaches for pixel-level and instance-level semantic labeling. Cityscapes is comprised of a large, diverse set of stereo video sequences recorded in streets from 50 different cities. 5000 of these images have high quality pixel-level annotations, 20 000 additional images have coarse annotations to enable methods that leverage large volumes of weakly-labeled data. Crucially, our effort exceeds previous attempts in terms of dataset size, annotation richness, scene variability, and complexity. Our accompanying empirical study provides an in-depth analysis of the dataset characteristics, as well as a performance evaluation of several state-of-the-art approaches based on our benchmark.

https://www.cityscapes-dataset.com/wordpress/wp-content/papercite-data/pdf/cordts2016cityscapes.pdf

The Cityscapes Dataset for Semantic Urban Scene Understanding

We present a novel dataset captured from a VW station wagon for use in mobile robotics and autonomous driving research. In total, we recorded 6 hours of traffic scenarios at 10-100 Hz using a variety of sensor modalities such as high-resolution color and grayscale stereo cameras, a Velodyne 3D laser scanner and a high-precision GPS/IMU inertial navigation system. The scenarios are diverse, capturing real-world traffic situations, and range from freeways over rural areas to inner-city scenes with many static and dynamic objects. Our data is calibrated, synchronized and timestamped, and we provide the rectified and raw image sequences. Our dataset also contains object labels in the form of 3D tracklets, and we provide online benchmarks for stereo, optical flow, object detection and other tasks. This paper describes our recording platform, the data format and the utilities that we provide.

/pdf/vision-meets-robotics-the-kitti-dataset-2h3kd0lpgn.pdf

Vision meets robotics: The KITTI dataset

Building good 3D maps is a challenging and expensive task, which requires high-quality sensors and careful, time-consuming scanning. We seek to reduce the cost of building good reconstructions by correcting views of existing low-quality ones in a post-hoc fashion using learnt priors over surfaces and appearance. We train a CNN model to predict the difference in inverse-depth from varying viewpoints of two meshes -- one of low quality that we wish to correct, and one of high-quality that we use as a reference. 
In contrast to previous work, we pay attention to the problem of excessive smoothing in corrected meshes. We address this with a suitable network architecture, and introduce a loss-weighting mechanism that emphasises edges in the prediction. Furthermore, smooth predictions result in geometrical inconsistencies. To deal with this issue, we present a loss function which penalises re-projection differences that are not due to occlusions. Our model reduces gross errors by 45.3%--77.5%, up to five times more than previous work.

Learning Geometrically Consistent Mesh Corrections.

One of the main challenges for sustained flight of current aerial micro robots is the low energy density available from common power sources. In this paper we propose solid rocket fuel powered micro thrusters as a high energy density actuation method for aerial micro robots. In a multi stage configuration these thrusters can be used for intermittent flight which can decrease the energetic cost of locomotion. In particular we focus on the fabrication method and characterization of multistage micro thrusters with a diameter of 3mm and 6.4mm. We demonstrate a sustained and repeatable thrust force of up to 35mN for a duration of up to 42s and a multi-stage designs with a time delay of up to 4.7s between the propulsion phases. Furthermore, we present a take-off trajectory of a 10cm rocket glider with an integrated micro thruster as propulsion mechanism showing that the technologies developed can be used to successfully power micro robots in flight. Future work will focus on control and flight dynamics of micro thruster powered gliders. Wider applications of similar thrusters can include other robotic applications where low weight and high force is important such as for jumping or running robots.1

Multi-Stage Micro Rockets for Robotic Insects

Our goal is to develop an automated digital fabrication process that can make any object out of smart materials. In this paper, we present an algorithm for creating shapes by the process of duplication, using modules we have termed smart sand. The object to be duplicated is dipped into a bag of smart sand; the particles exchange messages to sense the ' shape; and then the particles selectively form mechanical bonds with their neighbors to form a duplicate of the original. Our algorithm is capable of duplicating convex and concave 3D objects in a completely distributed manner. It uses O(1) storage space and O(n) inter-module messages per module. We perform close to 500 experiments using a realistic simulator with over 1400 modules. These experiments confirm the functionality and messaging demands of our distributed duplication algorithm while demonstrating that the algorithm can be used to form interesting and useful shapes.

What's in the Bag: A Distributed Approach to 3D Shape Duplication with Modular Robots

Navigation data is generated by receiving 502 a new experience data set (321,Fig.3) relating to a new experience (image) capture. At least one stored experience data set (320,Fig.3) relating to at least one previous experience (image) capture is also received 504. An experience data set includes a set of nodes, with each node comprising a series of visual image frames taken over a series of time frames. A candidate set of said nodes is obtained 506 from the stored experience data set that potentially matches a said node in the new experience data set, and then a check 508 is performed to see if a node in the candidate set matches the node in the new experience data set. If the result of the checking is positive then data relating to the matched nodes is added 510 to a place data set useable for navigation, the place data set indicating that said nodes in different said experience data sets relate to a same place. The new experience set may comprise data received from a sensor (302,Fig.3) such as a stereoscopic camera operating onboard a vehicle (306,Fig.3).

Generating navigation data in long term autonomous robotic systems

In this work, we train a network to simultaneously perform segmentation and pixel-wise Out-of-Distribution (OoD) detection, such that the segmentation of unknown regions of scenes can be rejected. This is made possible by leveraging an OoD dataset with a novel contrastive objective and data augmentation scheme. By combining data including unknown classes in the training data, a more robust feature representation can be learned with known classes represented distinctly from those unknown. When presented with unknown classes or conditions, many current approaches for segmentation frequently exhibit high confidence in their inaccurate segmentations and cannot be trusted in many operational environments. We validate our system on a real-world dataset of unusual driving scenes, and show that by selectively segmenting scenes based on what is predicted as OoD, we can increase the segmentation accuracy by an IoU of 0.2 with respect to alternative techniques.

Paul Newman

Papers

Learning Geometrically Consistent Mesh Corrections.

Multi-Stage Micro Rockets for Robotic Insects

What's in the Bag: A Distributed Approach to 3D Shape Duplication with Modular Robots

Generating navigation data in long term autonomous robotic systems

Fool Me Once: Robust Selective Segmentation via Out-of-Distribution Detection with Contrastive Learning