Because there are various unsafe factors on the road, the testing of the virtual environment is an important part of the automatic driving technology. This paper presents a CARLA vehicle and its distance detection system in a virtual environment. Based on the existing Yolo v5s neural network structure, this paper proposes a new neural network structure Yolo v5-Ghost. Adjusted the network layer structure of Yolo v5s. The computational complexity is reduced, and the proposed neural network structure is more suitable for embedded devices. After testing the new network structure, the detection accuracy of Yolo v5s is 83.36%mAP(mean Average Precision), the detection speed is 28.57FPS（Frames Per Second）, and the detection accuracy of Yolo v5-Ghost is 80.76%mAP, the detection speed is 47.62FPS. The paper also detects the vehicle distance based on the pictures obtained by the monocular camera in the CARLA virtual environment. The detected distance error is about 5% on average.

Real-Time Vehicle and Distance Detection Based on Improved Yolo v5 Network

Due to the characteristics of time-varying traffic and nonlinearity, the short-term traffic flow data are difficult to predict accurately. The purpose of this paper is to improve the short-term traffic flow prediction accuracy through the proposed improved wavelet neural network prediction model and provide basic data and decision support for the intelligent traffic management system. In view of the extremely strong nonlinear processing power, self-organization, self-adaptation and learning ability of wavelet neural network (WNN), this paper uses it as the basic prediction model and uses the particle swarm optimization algorithm for the slow convergence rate and local optimal problem of WNN prediction algorithm. With the advantages of fast convergence, high robustness and strong global search ability, an improved particle swarm optimization algorithm is proposed to optimize the wavelet neural network prediction model. The improved wavelet neural network is used to predict short-term traffic flow. The experimental results show that the proposed algorithm is more efficient than the WNN and PSO–WNN algorithms alone. The prediction results are more stable and more accurate. Compared with the traditional wavelet neural network, the error is reduced by 14.994%.

Short-term traffic flow prediction based on improved wavelet neural network

Recently, the notion of a smart city, which includes smart well‐being, smart transit, and smart society, has attracted much attention due to its impact on people's quality of living. Data in smart cities are characterized by variety, velocity, volume, value, and veracity that are the well‐known characteristics of big data. The fast pace expanding of IoT devices and sensors in smart cities generates a huge volume of data that can help decision‐makers and managers in city management. The aim of this article is to wholly and systematically review big data handling approaches in smart cities, in which we analyze research efforts published between 2013 and February 2021, where these techniques are categorized based on their algorithms and architectures. Further, the main ideas, evaluation techniques, tools, evaluation metrics, algorithm types, advantages, and disadvantages are explored. Additionally, essential evaluation factors are introduced in which scalability and availability by 16%, time by 15% and accuracy by 11% are more in focus, and finally, some of the challenges, open issues, and future trends that are valuable for further research are suggested in big data handling approaches in smart cities.

Leveraging big data in smart cities: A systematic review

With increasing progress in autonomous driving, the human does not have to be in control of the vehicle for the entire drive. A human driver obtains the control of the vehicle in case of an autonomous system failure or when the vehicle encounters an unknown traffic situation it cannot handle on its own. A critical part of this transition to human control is to ensure a sufficient driver situation awareness. Currently, no direct method to explicitly estimate driver awareness exists. In this paper, we propose a novel system to explicitly measure the situation awareness of the driver. Our approach is inspired by methods used in aviation. However, in contrast to aviation, the situation awareness in driving is determined by the detection and understanding of dynamically changing and previously unknown situation elements. Our approach uses machine learning to define the best possible situation awareness. We also propose to measure the actual situation awareness of the driver using eye tracking. Comparing the actual awareness to the target awareness allows us to accurately assess the awareness the driver has of the current traffic situation. To test our approach, we conducted a user study. We measured the situation awareness score of our model for 8 unique traffic scenarios. The results experimentally validate the accuracy of the proposed driver awareness model.

Measuring Driver Situation Awareness Using Region-of-Interest Prediction and Eye Tracking

Visual environment perception is one of the key elements for autonomous and manual driving. Modern fully automated vehicles are equipped with a range of different sensors and capture the surroundings with multiple cameras. The ability to predict human driver’s attention is the basis for various autonomous driving functions. State-of-the-art attention prediction approaches use only a single front facing camera and rely on automatically generated training data. In this paper, we present a manually labeled multi-view region of interest dataset. We use our dataset to finetune a state-of-the-art region of interest prediction model for multiple camera views. Additionally, we show that using two separate models focusing on either front or rear view data improves the region of interest prediction. We further propose a semi-supervised annotation framework which uses the best performing finetuned models for generating pseudo labels to improve the efficiency of the labeling process. Our results show that existing region of interest prediction performs well on front view data, but finetuning improves the performance especially for rear view data. Our current dataset consists of about 16000 images and we plan to further increase the size of the dataset. The dataset and the source code of the proposed semi-supervised annotation framework will be made available on GitHub and can be used to generate custom region of interest data.

Multi-View Region of Interest Prediction for Autonomous Driving Using Semi-Supervised Labeling

Teledriving is a possible fallback mode to cope with failures of fully autonomous vehicles. One important requirement for teleoperated vehicles is a reliable low delay data transmission solution, which adapts to the current network conditions to provide the operator with the best possible situation awareness. Currently, there is no easily accessible solution for the evaluation of such systems and algorithms in a fully controllable environment available. To this end we propose an open source framework for teleoperated driving research using low-cost off-the-shelf components. The proposed system is an extension of the open source simulator CARLA, which is responsible for rendering the driving environment and providing reproducible scenario evaluation. As a proof of concept, we evaluated our teledriving solution against CARLA in remote and local driving scenarios. The proposed teledriving system leads to almost identical performance measurements for local and remote driving. In contrast, remote driving using CARLA's client server communication results in drastically reduced operator performance. Further, the framework provides an interface for the adaptation of the temporal resolution and target bitrate of the compressed video streams. The proposed framework reduces the required setup effort for teleoperated driving research in academia and industry.

TELECARLA: An Open Source Extension of the CARLA Simulator for Teleoperated Driving Research Using Off-the-Shelf Components

We present an introspective failure prediction approach for autonomous vehicles. In autonomous driving, complex or unknown scenarios can cause a disengagement of the self-driving system. Disengagements can be triggered either by automatic safety measures or by human intervention. We propose to use recorded disengagement sequences from test drives as training data to learn to predict future failures. The system then learns introspectively from its own previous mistakes. In order to predict failures as early as possible, we propose a machine learning approach where sequences of sensor data are classified as either failure or success. The car itself is treated as a black box. Our method combines two sensor modalities that contain different types of information. An image-based model learns to detect generally challenging situations such as crowded intersections accurately multiple seconds in advance. A state data based model allows to detect fast changes immediately before a failure, such as sudden braking or swerving. The outcome of the individual models is fused by averaging the individual failure probabilities. We evaluate our approach on a data set provided by the BMW Group containing 14 hours of autonomous driving. The proposed late fusion approach allows for predicting failures at an accuracy of more than 85% seven seconds in advance, at a false positive rate of 20%. The proposed method outperforms state-of-the-art failure prediction by more than 15% while being a flexible framework that allows for straightforward addition of further sensor modalities.

Introspective Failure Prediction for Autonomous Driving Using Late Fusion of State and Camera Information

Failures in autonomous driving caused by complex traffic situations or model inaccuracies remain inevitable in the near future. While much research is focused on how to prevent such failures, comparatively little research has been done on predicting them. An early failure prediction would allow for more time to take actions to resolve challenging situations. In this work, we propose an introspective approach to predict future disengagements of the car by learning from previous disengagement sequences. Our method is designed to detect failures as early as possible by using sensor data from up to ten seconds before each disengagement. The car itself is treated as a black box, with only its state data and the number of detected objects being required. Since no model-specific knowledge is needed, our method is applicable to any self-driving system. Currently, no public data of real-life disengagements is available. To test our approach, we therefore use autonomous driving data provided by BMW that was collected with BMW research vehicles over multiple months. We show that an LSTM classifier trained with sequences of state data can predict failures up to seven seconds in advance with an accuracy of more than 80%. This is two seconds earlier than comparable approaches from the literature.

Christopher B. Kuhn

Papers

TELECARLA: An Open Source Extension of the CARLA Simulator for Teleoperated Driving Research Using Off-the-Shelf Components

Introspective Failure Prediction for Autonomous Driving Using Late Fusion of State and Camera Information

Introspective Black Box Failure Prediction for Autonomous Driving

Measuring Driver Situation Awareness Using Region-of-Interest Prediction and Eye Tracking

Multi-View Region of Interest Prediction for Autonomous Driving Using Semi-Supervised Labeling