Showing papers in "IEEE Transactions on Intelligent Transportation Systems in 2021"

Deep Reinforcement Learning for Autonomous Driving: A Survey

Abstract: With the development of deep representation learning, the domain of reinforcement learning (RL) has become a powerful learning framework now capable of learning complex policies in high dimensional environments. This review summarises deep reinforcement learning (DRL) algorithms and provides a taxonomy of automated driving tasks where (D)RL methods have been employed, while addressing key computational challenges in real world deployment of autonomous driving agents. It also delineates adjacent domains such as behavior cloning, imitation learning, inverse reinforcement learning that are related but are not classical RL algorithms. The role of simulators in training agents, methods to validate, test and robustify existing solutions in RL are discussed.

Deep Multi-Modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challenges

Abstract: Recent advancements in perception for autonomous driving are driven by deep learning. In order to achieve robust and accurate scene understanding, autonomous vehicles are usually equipped with different sensors (e.g. cameras, LiDARs, Radars), and multiple sensing modalities can be fused to exploit their complementary properties. In this context, many methods have been proposed for deep multi-modal perception problems. However, there is no general guideline for network architecture design, and questions of “what to fuse”, “when to fuse”, and “how to fuse” remain open. This review paper attempts to systematically summarize methodologies and discuss challenges for deep multi-modal object detection and semantic segmentation in autonomous driving. To this end, we first provide an overview of on-board sensors on test vehicles, open datasets, and background information for object detection and semantic segmentation in autonomous driving research. We then summarize the fusion methodologies and discuss challenges and open questions. In the appendix, we provide tables that summarize topics and methods. We also provide an interactive online platform to navigate each reference: https://boschresearch.github.io/multimodalperception/ .

A Survey of Deep Learning Applications to Autonomous Vehicle Control

Abstract: Designing a controller for autonomous vehicles capable of providing adequate performance in all driving scenarios is challenging due to the highly complex environment and inability to test the system in the wide variety of scenarios which it may encounter after deployment. However, deep learning methods have shown great promise in not only providing excellent performance for complex and non-linear control problems, but also in generalising previously learned rules to new scenarios. For these reasons, the use of deep learning for vehicle control is becoming increasingly popular. Although important advancements have been achieved in this field, these works have not been fully summarised. This paper surveys a wide range of research works reported in the literature which aim to control a vehicle through deep learning methods. Although there exists overlap between control and perception, the focus of this paper is on vehicle control, rather than the wider perception problem which includes tasks such as semantic segmentation and object detection. The paper identifies the strengths and limitations of available deep learning methods through comparative analysis and discusses the research challenges in terms of computation, architecture selection, goal specification, generalisation, verification and validation, as well as safety. Overall, this survey brings timely and topical information to a rapidly evolving field relevant to intelligent transportation systems.

Intelligent Edge Computing in Internet of Vehicles: A Joint Computation Offloading and Caching Solution

Abstract: Recently, Internet of Vehicles (IoV) has become one of the most active research fields in both academic and industry, which exploits resources of vehicles and Road Side Units (RSUs) to execute various vehicular applications. Due to the increasing number of vehicles and the asymmetrical distribution of traffic flows, it is essential for the network operator to design intelligent offloading strategies to improve network performance and provide high-quality services for users. However, the lack of global information and the time-variety of IoVs make it challenging to perform effective offloading and caching decisions under long-term energy constraints of RSUs. Since Artificial Intelligence (AI) and machine learning can greatly enhance the intelligence and the performance of IoVs, we push AI inspired computing, caching and communication resources to the proximity of smart vehicles, which jointly enable RSU peer offloading, vehicle-to-RSU offloading and content caching in the IoV framework. A Mix Integer Non-Linear Programming (MINLP) problem is formulated to minimize total network delay, consisting of communication delay, computation delay, network congestion delay and content downloading delay of all users. Then, we develop an online multi-decision making scheme (named OMEN) by leveraging Lyapunov optimization method to solve the formulated problem, and prove that OMEN achieves near-optimal performance. Leveraging strong cognition of AI, we put forward an imitation learning enabled branch-and-bound solution in edge intelligent IoVs to speed up the problem solving process with few training samples. Experimental results based on real-world traffic data demonstrate that our proposed method outperforms other methods from various aspects.

Deep Learning for Visual Tracking: A Comprehensive Survey

Abstract: Visual target tracking is one of the most sought-after yet challenging research topics in computer vision. Given the ill-posed nature of the problem and its popularity in a broad range of real-world scenarios, a number of large-scale benchmark datasets have been established, on which considerable methods have been developed and demonstrated with significant progress in recent years - predominantly by recent deep learning (DL)-based methods. This survey aims to systematically investigate the current DL-based visual tracking methods, benchmark datasets, and evaluation metrics. It also extensively evaluates and analyzes the leading visual tracking methods. First, the fundamental characteristics, primary motivations, and contributions of DL-based methods are summarized from nine key aspects of: network architecture, network exploitation, network training for visual tracking, network objective, network output, exploitation of correlation filter advantages, aerial-view tracking, long-term tracking, and online tracking. Second, popular visual tracking benchmarks and their respective properties are compared, and their evaluation metrics are summarized. Third, the state-of-the-art DL-based methods are comprehensively examined on a set of well-established benchmarks of OTB2013, OTB2015, VOT2018, LaSOT, UAV123, UAVDT, and VisDrone2019. Finally, by conducting critical analyses of these state-of-the-art trackers quantitatively and qualitatively, their pros and cons under various common scenarios are investigated. It may serve as a gentle use guide for practitioners to weigh when and under what conditions to choose which method(s). It also facilitates a discussion on ongoing issues and sheds light on promising research directions.

A Hybrid Deep Learning Model With Attention-Based Conv-LSTM Networks for Short-Term Traffic Flow Prediction

Abstract: Accurate short-time traffic flow prediction has gained gradually increasing importance for traffic plan and management with the deployment of intelligent transportation systems (ITSs). However, the existing approaches for short-term traffic flow prediction are unable to efficiently capture the complex nonlinearity of traffic flow, which provide unsatisfactory prediction accuracy. In this paper, we propose a deep learning based model which uses hybrid and multiple-layer architectures to automatically extract inherent features of traffic flow data. Firstly, built on the convolutional neural network (CNN) and the long short-term memory (LSTM) network, we develop an attention-based Conv-LSTM module to extract the spatial and short-term temporal features. The attention mechanism is properly designed to distinguish the importance of flow sequences at different times by automatically assigning different weights. Secondly, to further explore long-term temporal features, we propose a bidirectional LSTM (Bi-LSTM) module to extract daily and weekly periodic features so as to capture variance tendency of the traffic flow from both previous and posterior directions. Finally, extensive experimental results are presented to show that the proposed model combining the attention Conv-LSTM and Bi-LSTM achieves better prediction performance compared with other existing approaches.

V2VR: Reliable Hybrid-Network-Oriented V2V Data Transmission and Routing Considering RSUs and Connectivity Probability

Abstract: Vehicular ad hoc networks (VANETs) have been widely used in intelligent transportation systems (ITSs) for purposes such as the control of unmanned aerial vehicles (UAVs) and trajectory prediction. However, an efficient and reliable data routing decision scheme is critical for VANETs due to the feature of self-organizing wireless multi-hop communication. Compared with wireless networks, which are unstable and have limited bandwidth, wired networks normally provide longer transmission distances, higher network speeds and greater reliability. To address this problem, this paper proposes a reliable VANET routing decision scheme based on the Manhattan mobility model, which considers the integration of roadside units (RSUs) into wireless and wired modes for data transmission and routing optimization. First, the problems of frequently moving vehicles and network connectivity are analyzed based on road networks and the motion information of vehicle nodes. Second, an improved greedy algorithm for vehicle wireless communication is used for network optimization, and a wired RSU network is also applied. In addition, routing decision analysis is carried out in accordance with the probabilistic model for various transmission ranges by checking the connectivity among vehicles and RSUs. Finally, comprehensive experiments show that our proposed method can support real-time planning and improve network transmission performance compared with other baseline protocol approaches in terms of several metrics, including package delivery ratio, time delay and wireless hops.

User-Oriented Virtual Mobile Network Resource Management for Vehicle Communications

Abstract: Currently, advanced communications and networks greatly enhance user experiences and have a major impact on all aspects of people’s lifestyles in terms of work, society, and the economy. However improving competitiveness and sustainable vehicle network services, such as higher user experience, considerable resource utilization and effective personalized services, is a great challenge. Addressing these issues, this paper proposes a virtual network resource management based on user behavior to further optimize the existing vehicle communications. In particular, ensemble learning is implemented in the proposed scheme to predict the user’s voice call duration and traffic usage for supporting user-centric mobile services optimization. Sufficient experiments show that the proposed scheme can significantly improve the quality of services and experiences and that it provides a novel idea for optimizing vehicle networks.

An Edge Traffic Flow Detection Scheme Based on Deep Learning in an Intelligent Transportation System

Abstract: An intelligent transportation system (ITS) plays an important role in public transport management, security and other issues. Traffic flow detection is an important part of the ITS. Based on the real-time acquisition of urban road traffic flow information, an ITS provides intelligent guidance for relieving traffic jams and reducing environmental pollution. The traffic flow detection in an ITS usually adopts the cloud computing mode. The edge of the network will transmit all the captured video to the cloud computing center. However, the increasing traffic monitoring has brought great challenges to the storage, communication and processing of traditional transportation systems based on cloud computing. To address this issue, a traffic flow detection scheme based on deep learning on the edge node is proposed in this article. First, we propose a vehicle detection algorithm based on the YOLOv3 (You Only Look Once) model trained with a great volume of traffic data. We pruned the model to ensure its efficiency on the edge equipment. After that, the DeepSORT (Deep Simple Online and Realtime Tracking) algorithm is optimized by retraining the feature extractor for multiobject vehicle tracking. Then, we propose a real-time vehicle tracking counter for vehicles that combines the vehicle detection and vehicle tracking algorithms to realize the detection of traffic flow. Finally, the vehicle detection network and multiple-object tracking network are migrated and deployed on the edge device Jetson TX2 platform, and we verify the correctness and efficiency of our framework. The test results indicate that our model can efficiently detect the traffic flow with an average processing speed of 37.9 FPS (frames per second) and an average accuracy of 92.0% on the edge device.

Deep Learning-Based Traffic Safety Solution for a Mixture of Autonomous and Manual Vehicles in a 5G-Enabled Intelligent Transportation System

Abstract: It is expected that a mixture of autonomous and manual vehicles will persist as a part of the intelligent transportation system (ITS) for many decades. Thus, addressing the safety issues arising from this mix of autonomous and manual vehicles before autonomous vehicles are entirely popularized is crucial. As the ITS system has increased in complexity, autonomous vehicles exhibit problems such as a low intention recognition rate and poor real-time performance when predicting the driving direction; these problems seriously affect the safety and comfort of mixed traffic systems. Therefore, the ability of autonomous vehicles to predict the driving direction in real time according to the surrounding traffic environment must be improved and researchers must work to create a more mature ITS. In this paper, we propose a deep learning-based traffic safety solution for a mixture of autonomous and manual vehicles in a 5G-enabled ITS. In this scheme, a driving trajectory dataset and a natural-driving dataset are employed as the network inputs to long-term memory networks in the 5G-enabled ITS: the probability matrix of each intention is calculated by the softmax function. Then, the final intention probability is obtained by fusing the mean rule in the decision layer. Experimental results show that the proposed scheme achieves intention recognition rates of 91.58% and 90.88% for left and right lane changes, respectively, effectively improving both accuracy and real-time intention recognition and improving the lane change problem in a mixed traffic environment.

Optimized Graph Convolution Recurrent Neural Network for Traffic Prediction

Abstract: Traffic prediction is a core problem in the intelligent transportation system and has broad applications in the transportation management and planning, and the main challenge of this field is how to efficiently explore the spatial and temporal information of traffic data. Recently, various deep learning methods, such as convolution neural network (CNN), have shown promising performance in traffic prediction. However, it samples traffic data in regular grids as the input of CNN, thus it destroys the spatial structure of the road network. In this paper, we introduce a graph network and propose an optimized graph convolution recurrent neural network for traffic prediction, in which the spatial information of the road network is represented as a graph. Additionally, distinguishing with most current methods using a simple and empirical spatial graph, the proposed method learns an optimized graph through a data-driven way in the training phase, which reveals the latent relationship among the road segments from the traffic data. Lastly, the proposed method is evaluated on three real-world case studies, and the experimental results show that the proposed method outperforms state-of-the-art traffic prediction methods.

A Hierarchical Blockchain-Enabled Federated Learning Algorithm for Knowledge Sharing in Internet of Vehicles

Abstract: Internet of Vehicles (IoVs) is highly characterized by collaborative environment data sensing, computing and processing. Emerging big data and Artificial Intelligence (AI) technologies show significant advantages and efficiency for knowledge sharing among intelligent vehicles. However, it is challenging to guarantee the security and privacy of knowledge during the sharing process. Moreover, conventional AI-based algorithms cannot work properly in distributed vehicular networks. In this paper, a hierarchical blockchain framework and a hierarchical federated learning algorithm are proposed for knowledge sharing, by which vehicles learn environmental data through machine learning methods and share the learning knowledge with each others. The proposed hierarchical blockchain framework is feasible for the large scale vehicular networks. The hierarchical federated learning algorithm is designed to meet the distributed pattern and privacy requirement of IoVs. Knowledge sharing is then modeled as a trading market process to stimulate sharing behaviours, and the trading process is formulated as a multi-leader and multi-player game. Simulation results show that the proposed hierarchical algorithm can improve the sharing efficiency and learning quality. Furthermore, the blockchain-enabled framework is able to deal with certain malicious attacks effectively.

Research Advances and Challenges of Autonomous and Connected Ground Vehicles

Abstract: Autonomous vehicle (AV) technology can provide a safe and convenient transportation solution for the public, but the complex and various environments in the real world make it difficult to operate safely and reliably. A connected autonomous vehicle (CAV) is an AV with vehicle connectivity capability, which enhances the situational awareness of the AV and enables the cooperation between AVs. Hence, CAV technology can enhance the capabilities and robustness of AV to be a promising transportation solution in the future. This paper introduces a representative architecture of CAVs and surveys the latest research advances, methods, and algorithms for sensing, perception, planning, and control of CAVs. It reviews the state-of-the-art and state-of-the-practice (when applicable) of a multi-layer Perception-Planning-Control architecture including on-board sensors and vehicular communications, the methods of sensor fusion and localization and mapping in the perception layer, the algorithms of decision making and trajectory planning in the planning layer, and the control strategies of trajectory tracking in the control layer. Furthermore, the implementations and impact of vehicle connectivity and the corresponding consequential challenges of cooperative perception, complex connected decision making, and multi-vehicle controls are summarized and their significant research issues enumerated. Most importantly, the critical review in this paper provides a list and discussion of the remaining challenges and unsolved problems of CAVs in each Section which would be helpful to researchers in the field. The comprehensive coverage of this paper makes it particularly useful to academic researchers, practitioners, and students alike.

Efficient and Secure Routing Protocol Based on Artificial Intelligence Algorithms With UAV-Assisted for Vehicular Ad Hoc Networks in Intelligent Transportation Systems

Abstract: Vehicular Ad hoc Networks (VANETs) that are considered as a subset of Mobile Ad hoc Networks (MANETs) can be applied in the field of transportation especially in Intelligent Transportation Systems (ITS). The routing process in these networks is a challenging task due to rapid topology changes, high vehicle mobility and frequent disconnection of links. Therefore, developing an efficient routing protocol that satisfies restriction of delay and minimum overhead is faced with many difficulties and limitations. Also, the detection of malicious vehicles is a significant task in VANETs. To address these issues, using Unmanned Aerial Vehicles (UAVs) can be helpful to cope with these limitations. In this paper, operation of UAVs in ad hoc mode and their cooperation with vehicles in VANETs are studied to help in the process of routing and detection of malicious vehicles. A routing protocol named VRU is proposed that includes two distinct ways of routing of data: (1) delivering packets of data between vehicles with the help of UAVs using a protocol named VRU_vu, and (2) routing packet of data between UAVs using a protocol named VRU_u. The NS-2.35 simulator under Linux Ubuntu 12.04 is utilized in order to appraise the performance of VRU routing components in an urban scenario. Also, VanetMobiSim generator of mobility and MobiSim are used to produce the motions of vehicles and to produce the motions of UAVs, respectively. The performance analysis displays that VRU protocol can improve the packet delivery ratio by 16% and detection ratio by 7% compared to other reviewed routing protocol. Also, VRU protocol decreases end-to-end delay by an average of 13% and overhead by 40%.

Joint Computing and Caching in 5G-Envisioned Internet of Vehicles: A Deep Reinforcement Learning-Based Traffic Control System

Abstract: Recent developments of edge computing and content caching in wireless networks enable the Intelligent Transportation System (ITS) to provide high-quality services for vehicles. However, a variety of vehicular applications and time-varying network status make it challenging for ITS to allocate resources efficiently. Artificial intelligence algorithms, owning the cognitive capability for diverse and time-varying features of Internet of Connected Vehicles (IoCVs), enable an intent-based networking for ITS to tackle the above-mentioned challenges. In this paper, we develop an intent-based traffic control system by investigating Deep Reinforcement Learning (DRL) for 5G-envisioned IoCVs, which can dynamically orchestrate edge computing and content caching to improve the profits of Mobile Network Operator (MNO). By jointly analyzing MNO’s revenue and users’ quality of experience, we define a profit function to calculate the MNO’s profits. After that, we formulate a joint optimization problem to maximize MNO’s profits, and develop an intelligent traffic control scheme by investigating DRL, which can improve system profits of the MNO and allocate network resources effectively. Experimental results based on real traffic data demonstrate our designed system is efficient and well-performed.

Deep Learning for Safe Autonomous Driving: Current Challenges and Future Directions

Abstract: Advances in information and signal processing technologies have a significant impact on autonomous driving (AD), improving driving safety while minimizing the efforts of human drivers with the help of advanced artificial intelligence (AI) techniques. Recently, deep learning (DL) approaches have solved several real-world problems of complex nature. However, their strengths in terms of control processes for AD have not been deeply investigated and highlighted yet. This survey highlights the power of DL architectures in terms of reliability and efficient real-time performance and overviews state-of-the-art strategies for safe AD, with their major achievements and limitations. Furthermore, it covers major embodiments of DL along the AD pipeline including measurement, analysis, and execution, with a focus on road, lane, vehicle, pedestrian, drowsiness detection, collision avoidance, and traffic sign detection through sensing and vision-based DL methods. In addition, we discuss on the performance of several reviewed methods by using different evaluation metrics, with critics on their pros and cons. Finally, this survey highlights the current issues of safe DL-based AD with a prospect of recommendations for future research, rounding up a reference material for newcomers and researchers willing to join this vibrant area of Intelligent Transportation Systems.

Towards Federated Learning in UAV-Enabled Internet of Vehicles: A Multi-Dimensional Contract-Matching Approach

Abstract: Coupled with the rise of Deep Learning, the wealth of data and enhanced computation capabilities of Internet of Vehicles (IoV) components enable effective Artificial Intelligence (AI) based models to be built. Beyond ground data sources, Unmanned Aerial Vehicles (UAVs) based service providers for data collection and AI model training, i.e., Drones-as-a-Service (DaaS), is becoming increasingly popular in recent years. However, the stringent regulations governing data privacy potentially impedes data sharing across independently owned UAVs. To this end, we propose the adoption of a Federated Learning (FL) based approach to enable privacy-preserving collaborative Machine Learning across a federation of independent DaaS providers for the development of IoV applications, e.g., for traffic prediction and car park occupancy management. Given the information asymmetry and incentive mismatches between the UAVs and model owners, we leverage on the self-revealing properties of a multi-dimensional contract to ensure truthful reporting of the UAV types, while accounting for the multiple sources of heterogeneity, e.g., in sensing, computation, and transmission costs. Then, we adopt the Gale-Shapley algorithm to match the lowest cost UAV to each subregion. The simulation results validate the incentive compatibility of our contract design, and shows the efficiency of our matching, thus guaranteeing profit maximization for the model owner amid information asymmetry.

A Clustering-Based Coverage Path Planning Method for Autonomous Heterogeneous UAVs

Abstract: Unmanned aerial vehicles (UAVs) have been widely applied in civilian and military applications due to their high autonomy and strong adaptability. Although UAVs can achieve effective cost reduction and flexibility enhancement in the development of large-scale systems, they result in a serious path planning and task allocation problem. Coverage path planning, which tries to seek flight paths to cover all of regions of interest, is one of the key technologies in achieving autonomous driving of UAVs and difficult to obtain optimal solutions because of its NP-Hard computational complexity. In this paper, we study the coverage path planning problem of autonomous heterogeneous UAVs on a bounded number of regions. First, with models of separated regions and heterogeneous UAVs, we propose an exact formulation based on mixed integer linear programming to fully search the solution space and produce optimal flight paths for autonomous UAVs. Then, inspired from density-based clustering methods, we design an original clustering-based algorithm to classify regions into clusters and obtain approximate optimal point-to-point paths for UAVs such that coverage tasks would be carried out correctly and efficiently. Experiments with randomly generated regions are conducted to demonstrate the efficiency and effectiveness of the proposed approach.

Mobility-Aware Proactive Edge Caching for Connected Vehicles Using Federated Learning

Abstract: Content Caching at the edge of vehicular networks has been considered as a promising technology to satisfy the increasing demands of computation-intensive and latency-sensitive vehicular applications for intelligent transportation. The existing content caching schemes, when used in vehicular networks, face two distinct challenges: 1) Vehicles connected to an edge server keep moving, making the content popularity varying and hard to predict. 2) Cached content is easily out-of-date since each connected vehicle stays in the area of an edge server for a short duration. To address these challenges, we propose a Mobility-aware Proactive edge Caching scheme based on Federated learning (MPCF). This new scheme enables multiple vehicles to collaboratively learn a global model for predicting content popularity with the private training data distributed on local vehicles. MPCF also employs a Context-aware Adversarial AutoEncoder to predict the highly dynamic content popularity. Besides, MPCF integrates a mobility-aware cache replacement policy, which allows the network edges to add/evict contents in response to the mobility patterns and preferences of vehicles. MPCF can greatly improve cache performance, effectively protect users’ privacy and significantly reduce communication costs. Experimental results demonstrate that MPCF outperforms other baseline caching schemes in terms of the cache hit ratio in vehicular edge networks.

Deep Learning on Traffic Prediction: Methods, Analysis and Future Directions

Abstract: Traffic prediction plays an essential role in intelligent transportation system. Accurate traffic prediction can assist route planing, guide vehicle dispatching, and mitigate traffic congestion. This problem is challenging due to the complicated and dynamic spatio-temporal dependencies between different regions in the road network. Recently, a significant amount of research efforts have been devoted to this area, especially deep learning method, greatly advancing traffic prediction abilities. The purpose of this paper is to provide a comprehensive survey on deep learning-based approaches in traffic prediction from multiple perspectives. Specifically, we first summarize the existing traffic prediction methods, and give a taxonomy. Second, we list the state-of-the-art approaches in different traffic prediction applications. Third, we comprehensively collect and organize widely used public datasets in the existing literature to facilitate other researchers. Furthermore, we give an evaluation and analysis by conducting extensive experiments to compare the performance of different methods on a real-world public dataset. Finally, we discuss open challenges in this field.

Temporal Multi-Graph Convolutional Network for Traffic Flow Prediction

Abstract: Traffic flow prediction plays an important role in ITS (Intelligent Transportation System). This task is challenging due to the complex spatial and temporal correlations (e.g., the constraints of road network and the law of dynamic change with time). Existing work tried to solve this problem by exploiting a variety of spatiotemporal models. However, we observe that more semantic pair-wise correlations among possibly distant roads are also critical for traffic flow prediction. To jointly model the spatial, temporal, semantic correlations with various global features in the road network, this paper proposes T-MGCN ( Temporal Multi-Graph Convolutional Network ), a deep learning framework for traffic flow prediction. First, we identify several kinds of semantic correlations, and encode the non-Euclidean spatial correlations and heterogeneous semantic correlations among roads into multiple graphs. These correlations are then modeled by a multi-graph convolutional network. Second, a recurrent neural network is utilized to learn dynamic patterns of traffic flow to capture the temporal correlations. Third, a fully connected neural network is utilized to fuse the spatiotemporal correlations with global features. We evaluate T-MGCN on two real-world traffic datasets and observe improvement by approximately 3% to 6% as compared to the state-of-the-art baseline.

Topological Graph Convolutional Network-Based Urban Traffic Flow and Density Prediction

Abstract: With the development of modern Intelligent Transportation System (ITS), reliable and efficient transportation information sharing becomes more and more important. Although there are promising wireless communication schemes such as Vehicle-to-Everything (V2X) communication standards, information sharing in ITS still faces challenges such as the V2X communication overload when a large number of vehicles suddenly appeared in one area. This flash crowd situation is mainly due to the uncertainty of traffic especially in the urban areas during traffic rush hours and will significantly increase the V2X communication latency. In order to solve such flash crowd issues, we propose a novel system that can accurately predict the traffic flow and density in the urban area that can be used to avoid the V2X communication flash crowd situation. By combining the existing grid-based and graph-based traffic flow prediction methods, we use a Topological Graph Convolutional Network (ToGCN) followed with a Sequence-to-sequence (Seq2Seq) framework to predict future traffic flow and density with temporal correlations. The experimentation on a real-world taxi trajectory traffic data set is performed and the evaluation results prove the effectiveness of our method.

Resource Allocation in 5G IoV Architecture Based on SDN and Fog-Cloud Computing

Abstract: In the traditional cloud-based Internet of Vehicles (IoV) architecture, it is difficult to guarantee the low latency requirements of the current intelligent transportation system (ITS). As a supplement to cloud computing, fog computing can effectively alleviate the bottlenecks of cloud computing bandwidth and computing resources and improve the quality of service (QoS) of the IoV. However, as a distributed system that operates near users, fog computing has a complicated network structure. In the complex and dynamic IoV environment, to effectively manage these computing resources with different attributes and provide high-quality services, it is necessary to design an efficient architecture and a resource allocation algorithm. Therefore, on the basis of fog-cloud computing and software-defined networking (SDN), a novel 5G IoV architecture is designed. In addition, after fully considering the service requirements of the IoV, a model of four objectives is constructed, and a many-objective optimization algorithm is proposed. The experiment results show that the proposed algorithm outperforms the other state-of-the-art algorithms.

Trust in VANET: A Survey of Current Solutions and Future Research Opportunities

Abstract: Security and privacy will play a pivotal role in the commercialization of Vehicular Ad-hoc NETworks (VANETs). Traditionally, both cryptographic and non-cryptographic approaches have been used by researchers to address security and privacy issues and achieve secure Intelligent Transportation System (ITS) applications. However, some security goals such as trust and reputation, are still hard to achieve through conventional cryptographic approaches. Trust is the degree of certainty with which the received information is accepted and acted upon. Historically trust has been computed for both the content generator and the content itself with former known as entity trust and the latter known as data trust. Both entity and content trust are equally important to support trustworthy communication in VANET. We review, analyze, and compare some of the recently proposed trust establishment and management mechanisms (from 2014 to 2019) in vehicular networks. Furthermore, we also discuss the weaknesses and inadequacies of existing trust establishment and management approaches when deployed in a VANET environment. Finally, we discuss some future challenges that will need to be addressed for trustworthy communications in vehicular networks.

Testing Scenario Library Generation for Connected and Automated Vehicles, Part I: Methodology

Abstract: Testing and evaluation is a critical step in the development and deployment of connected and automated vehicles (CAVs), and yet there is no systematic framework to generate testing scenario library. This study aims to provide a general framework for the testing scenario library generation (TSLG) problem with different operational design domains (ODDs), CAV models, and performance metrics. Given an ODD, the testing scenario library is defined as a critical set of scenarios that can be used for CAV test. Each testing scenario is evaluated by a newly proposed measure, scenario criticality, which can be computed as a combination of maneuver challenge and exposure frequency. To search for critical scenarios, an auxiliary objective function is designed, and a multi-start optimization method along with seed-filling is applied. Theoretical analysis suggests that the proposed framework can obtain accurate evaluation results with much fewer number of tests, if compared with the on-road test method. In part II of the study, three case studies are investigated to demonstrate the proposed method. Reinforcement learning based technique is applied to enhance the searching method under high-dimensional scenarios.

How to Build a Graph-Based Deep Learning Architecture in Traffic Domain: A Survey

Abstract: In recent years, various deep learning architectures have been proposed to solve complex challenges (e.g. spatial dependency, temporal dependency) in traffic domain, which have achieved satisfactory performance. These architectures are composed of multiple deep learning techniques in order to tackle various challenges in traffic tasks. Traditionally, convolution neural networks (CNNs) are utilized to model spatial dependency by decomposing the traffic network as grids. However, many traffic networks are graph-structured in nature. In order to utilize such spatial information fully, it's more appropriate to formulate traffic networks as graphs mathematically. Recently, various novel deep learning techniques have been developed to process graph data, called graph neural networks (GNNs). More and more works combine GNNs with other deep learning techniques to construct an architecture dealing with various challenges in a complex traffic task, where GNNs are responsible for extracting spatial correlations in traffic network. These graph-based architectures have achieved state-of-the-art performance. To provide a comprehensive and clear picture of such emerging trend, this survey carefully examines various graph-based deep learning architectures in many traffic applications. We first give guidelines to formulate a traffic problem based on graph and construct graphs from various kinds of traffic datasets. Then we decompose these graph-based architectures to discuss their shared deep learning techniques, clarifying the utilization of each technique in traffic tasks. What's more, we summarize some common traffic challenges and the corresponding graph-based deep learning solutions to each challenge. Finally, we provide benchmark datasets, open source codes and future research directions in this rapidly growing field.

High-Resolution Vehicle Trajectory Extraction and Denoising From Aerial Videos

Abstract: In recent years, unmanned aerial vehicle (UAV) has become an increasingly popular tool for traffic monitoring and data collection on highways due to its advantage of low cost, high resolution, good flexibility, and wide spatial coverage. Extracting high-resolution vehicle trajectory data from aerial videos taken by a UAV flying over target highway segment becomes a critical research task for traffic flow modeling and analysis. This study aims at proposing a novel methodological framework for automatic and accurate vehicle trajectory extraction from aerial videos. The method starts by developing an ensemble detector to detect vehicles in the target region. Then, the kernelized correlation filter is applied to track vehicles fast and accurately. After that, a mapping algorithm is proposed to transform vehicle positions from the Cartesian coordinates in image to the Frenet coordinates to extract raw vehicle trajectories along the roadway curves. The data denoising is then performed using a wavelet transform to eliminate the biased vehicle trajectory positions. Our method is tested on two aerial videos taken on different urban expressway segments in both peak and non-peak hours on weekdays. The extracted vehicle trajectories are compared with manual calibrated data to testify the framework performance. The experimental results show that the proposed method successfully extracts vehicle trajectories with a high accuracy: the measurement error of Mean Squared Deviation is 2.301 m, the Root-mean-square deviation is 0.175 m, and the Pearson correlation coefficient is 0.999. The video and trajectory data in this study are publicly accessible for serving as benchmark at https://seutraffic.com .

Interpretable End-to-End Urban Autonomous Driving With Latent Deep Reinforcement Learning

Abstract: Unlike popular modularized framework, end-to-end autonomous driving seeks to solve the perception, decision and control problems in an integrated way, which can be more adapting to new scenarios and easier to generalize at scale. However, existing end-to-end approaches are often lack of interpretability, and can only deal with simple driving tasks like lane keeping. In this article, we propose an interpretable deep reinforcement learning method for end-to-end autonomous driving, which is able to handle complex urban scenarios. A sequential latent environment model is introduced and learned jointly with the reinforcement learning process. With this latent model, a semantic birdeye mask can be generated, which is enforced to connect with certain intermediate properties in today's modularized framework for the purpose of explaining the behaviors of learned policy. The latent space also significantly reduces the sample complexity of reinforcement learning. Comparison tests in a realistic driving simulator show that the performance of our method in urban scenarios with crowded surrounding vehicles dominates many baselines including DQN, DDPG, TD3 and SAC. Moreover, through masked outputs, the learned model is able to provide a better explanation of how the car reasons about the driving environment.

Human Trajectory Forecasting in Crowds: A Deep Learning Perspective

Abstract: Since the past few decades, human trajectory forecasting has been a field of active research owing to its numerous real-world applications: evacuation situation analysis, deployment of intelligent transport systems, traffic operations, to name a few. In this work, we cast the problem of human trajectory forecasting as learning a representation of human social interactions. Early works handcrafted this representation based on domain knowledge. However, social interactions in crowded environments are not only diverse but often subtle. Recently, deep learning methods have outperformed their handcrafted counterparts, as they learn about human-human interactions in a more generic data-driven fashion. In this work, we present an in-depth analysis of existing deep learning-based methods for modelling social interactions. We propose two domain-knowledge inspired data-driven methods to effectively capture these social interactions. To objectively compare the performance of these interaction-based forecasting models, we develop a large scale interaction-centric benchmark TrajNet++, a significant yet missing component in the field of human trajectory forecasting. We propose novel performance metrics that evaluate the ability of a model to output socially acceptable trajectories. Experiments on TrajNet++ validate the need for our proposed metrics, and our method outperforms competitive baselines on both real-world and synthetic datasets.

Automotive LiDAR Technology: A Survey

Abstract: Nowadays, and more than a decade after the first steps towards autonomous driving, we keep heading to achieve fully autonomous vehicles on our roads, with LiDAR sensors being a key instrument for the success of this technology. Such advances trigger the emergence of new players in the automotive industry, and along with car manufacturers, this sector represents a multibillion-dollar market where everyone wants to take a share. To understand recent advances and technologies behind LiDAR, this article presents a survey on LiDAR sensors for the automotive industry. With this work, we show the measurement principles and imaging techniques currently being used, going through a review of commercial systems and development solutions available in the market today. Furthermore, we highlight the current and future challenges, providing insights on how both research and industry can step towards better LiDAR solutions.