The PASCAL Visual Object Classes Challenge

Graphics processing units, GPUs, are powerful processors that can offer significant performance advantages over traditional CPUs. The last decade has seen rapid advancement in GPU computational power and generality. Recent technologies make it possible to use GPUs as co-processors to CPUs. The performance advantages of GPUs can be great, often outperforming traditional CPUs by orders of magnitude. While the motivations for developing systems with GPUs are clear, little research in the real-time systems field has been done to integrate GPUs into real-time multiprocessor systems. We present two real-time analysis methods, addressing real-world platform constraints, for such an integration into a soft real-time multiprocessor system and show that a GPU can be exploited to achieve greater levels of total system performance.

/pdf/globally-scheduled-real-time-multiprocessor-systems-with-1hzhy8c37u.pdf

Globally scheduled real-time multiprocessor systems with GPUs

We address the problem of traffic sign recognition in a light detection and ranging (LIDAR)-equipped vehicle. With the help of 3-D LIDAR points, the 2-D multiview sign images will be easily detected from the captured images of street signs. After detection, the sign recognition problem is formulated as a multiview object recognition task. We develop a metric-learning-based template matching approach for this task and learn a distance metric between the captured images and the corresponding sign templates. For each sign, recognition is done via soft voting by the recognition results of its corresponding multiview images. We propose a latent structural support vector machine (SVM)-based weakly supervised metric learning (WSMLR) method to learn the metric and a reliability classifier. The reliability classifier is used to determine each image's reliability, which serves as each image's weight in both the learning and soft voting procedure. We evaluate the proposed method for multiview traffic sign recognition on a multiview traffic sign data set with 112 categories and observe very encouraging results compared with other state-of-the-art methods. In addition, the method can be customized to solve the single-view sign recognition. The performance of our method for single-view sign recognition is tested on two public data sets, showing that our method is comparable with other competitive ones.

Weakly Supervised Metric Learning for Traffic Sign Recognition in a LIDAR-Equipped Vehicle

In recent years, Intelligent Transportation Systems (ITS) have seen efficient and faster development by implementing deep learning techniques in problem domains which were previously addressed using analytical or statistical solutions and also in some areas that were untouched. These improvements have facilitated traffic management and traffic planning, increased safety and security in transit roads, decreased costs of maintenance, optimized public transportation and ride-sharing company's performance, and advanced driver-less vehicle development to a new stage. This papers primary objective was to provide a review and comprehensive insight into the applications of deep learning models on intelligent transportation systems accompanied by presenting the progress of ITS research due to deep learning. First, different techniques of deep learning and their state-of-the-art are discussed, followed by an in-depth analysis and explanation of the current applications of these techniques in transportation systems. This enumeration of deep learning on ITS highlights its significance in the domain. The applications are furthermore categorized based on the gap they are trying to address. Finally, different embedded systems for deployment of these techniques are investigated and their advantages and weaknesses over each other are discussed. Based on this systematic review, credible benefits of deep learning models on ITS are demonstrated and directions for future research are discussed.

Applications of Deep Learning in Intelligent Transportation Systems

Graphics processing units (GPUs) are becoming increasingly important in today's platforms as their increased generality allows for them to be used as powerful coprocessors. In this paper, we explore possible applications for GPUs in real-time systems, discuss the limitations and constraints imposed by current GPU technology, and present a summary of our research addressing many such constraints.

/pdf/real-world-constraints-of-gpus-in-real-time-systems-5gssebs14g.pdf

Real-World Constraints of GPUs in Real-Time Systems

Generating annotations which can be used to train new models has become an independent field of research within machine learning. Its goal is producing highly accurate annotations as cost efficient as possible. 3D point clouds are the common sensor output when recording 3D data from a mobile platform. The latest ways of annotating 3D point clouds include their visualization on a 2D screen. This method contradicts the goal of time-efficient annotating since it is unintuitive and therefore unnecessarily time consuming. We present a novel labeling technique in Virtual Reality. Using our tool, we accelerate the process of data annotation significantly compared to existing approaches. Furthermore, we will give the machine learning community access to our tool and create a new community-labeled dataset for autonomous driving. Furthermore we plan to set up an annotation benchmark in which primarily commercial annotation companies but also researchers active in annotation can take part in. We present results from an experimental plattform based on Oculus Rift indicating a huge potential for VR annotations.

PointAtMe: Efficient 3D Point Cloud Labeling in Virtual Reality

We benchmark several widely-used deep learning frameworks and investigate the field programmable gate array (FPGA) deployment for performing traffic sign classification and detection. We evaluate the training speed and inference accuracy of these frameworks on the graphics processing unit (GPU) by training FPGA-deployment-suitable models with various input sizes on German Traffic Sign Recognition Benchmark (GTSRB), a traffic sign classification dataset. Then, selected trained classification models and various object detection models that we train on GTSRB's detection counterpart (i.e., German Traffic Sign Detection Benchmark) are evaluated with inference speed, accuracy, and FPGA power efficiency by varying different parameters such as floating-point precisions, batch sizes, etc. We discover that Neon and MXNet deliver the best training speed and classification accuracy on the GPU in general for all test cases, while TensorFlow is always among the frameworks with the highest inference accuracies. We observe that with the current OpenVINO release, the performance of lightweight models (e.g., MobileNet-v1-SSD, etc.) usually exceeds the requirement of real-time detection without losing much accuracy, while other models (e.g., VGG-SSD, ResNet-50-SSD) generally fail to do so. We also demonstrate that we can adjust the precision of bitstreams and the batch sizes to balance inference speed and accuracy of the applications deployed on the FPGA. Finally, we show that for all test cases, the FPGA always achieves higher power efficiency than the GPU.

/pdf/benchmarking-deep-learning-frameworks-and-investigating-fpga-ffaa60oi80.pdf

Benchmarking Deep Learning Frameworks and Investigating FPGA Deployment for Traffic Sign Classification and Detection

In this study we test the idea of using a graphics processing unit (GPU) as an embedded co-processor for real-time detection of European Union (EU) speed-limit signs. The input to the system is a set of grayscale videos recorded from a forward-facing camera mounted in a vehicle. We introduce a new technique for implementing the radial symmetry detector (RSD) efficiently using the native rendering capabilities of a GPU. The technique maps the algorithms to the hardware such that the detection of speed-limit sign candidates is significantly accelerated. The system reaches up to 88% detection rate and runs at 33 frames per second on video sequences with VGA (640×480) resolution on an embedded system with an Intel Atom 230 @ 1.67 GHz CPU and a NVIDIA GeForce 9400M GS GPU.

/pdf/feature-based-speed-limit-sign-detection-using-a-graphics-57e5ozlja5.pdf

Feature-based speed limit sign detection using a graphics processing unit

We present a template-based pipeline that performs realtime speed-limit-sign recognition using an embedded system with a lowend GPU as the main processing element. Our pipeline operates in the frequency domain, and uses nonlinear composite filters and a contrast-enhancing preprocessing step to improve its accuracy. Running at interactive rates, our system achieves 90% accuracy over 120 EU speed-limit signs on 45 minutes of video footage, superior to the 75% accuracy of a non-real-time GPU-based SIFT pipeline.

/pdf/a-template-based-approach-for-real-time-speed-limit-sign-2yti5d2wzf.pdf

A template-based approach for real-time speed-limit-sign recognition on an embedded system using GPU computing

Publisher Summary This chapter addresses the challenging problem of detecting and classifying speed limit signs in a real-time video stream using an embedded, low-end GPU. Graphics processing units (GPUs) have been increasingly used for applications beyond traditional graphics that are well suited for their capabilities. One of these fields is automotive computing. GPU-based techniques can be used for performing real-time speed-limit sign recognition on a resource-constrained system with a low-end GPU that can be embedded in a car. Three pipelines are implemented to address this problem. The first is a detection-only feature-based method that finds objects with radial symmetry. The second is a template-based method that searches for image templates in the frequency domain using Fast Fourier Transform (FFT) correlations, suitable for both EU and US speed-limit signs. The third is the classic GPU-based SIFT approach that provides a basis for evaluation of recognition results of the template-based approach. To provide fast runtimes and make efficient use of the underlying limited hardware resources, the inherent parallelism in the recognition process was exploited using data-parallel algorithms that are suitable for the GPU architecture. This study serves as proof of concept for the use of GPU computing in automotive tasks. However, in order to make the best use of an embedded GPU in the cars, it should be able to simultaneously run multiple other automotive tasks that are a good fit for the GPU architecture.

Jeffrey M. Ota

Papers

PointAtMe: Efficient 3D Point Cloud Labeling in Virtual Reality

Benchmarking Deep Learning Frameworks and Investigating FPGA Deployment for Traffic Sign Classification and Detection

Feature-based speed limit sign detection using a graphics processing unit

A template-based approach for real-time speed-limit-sign recognition on an embedded system using GPU computing

Real-Time Speed-Limit-Sign Recognition on an Embedded System Using a GPU