Driver decisions and behaviors are essential factors that can affect the driving safety. To understand the driver behaviors, a driver activities recognition system is designed based on the deep convolutional neural networks (CNN) in this paper. Specifically, seven common driving activities are identified, which are the normal driving, right mirror checking, rear mirror checking, left mirror checking, using in-vehicle radio device, texting, and answering the mobile phone, respectively. Among these activities, the first four are regarded as normal driving tasks, while the rest three are classified into the distraction group. The experimental images are collected using a low-cost camera, and ten drivers are involved in the naturalistic data collection. The raw images are segmented using the Gaussian mixture model to extract the driver body from the background before training the behavior recognition CNN model. To reduce the training cost, transfer learning method is applied to fine tune the pre-trained CNN models. Three different pre-trained CNN models, namely, AlexNet, GoogLeNet, and ResNet50 are adopted and evaluated. The detection results for the seven tasks achieved an average of 81.6% accuracy using the AlexNet, 78.6% and 74.9% accuracy using the GoogLeNet and ResNet50, respectively. Then, the CNN models are trained for the binary classification task and identify whether the driver is being distracted or not. The binary detection rate achieved 91.4% accuracy, which shows the advantages of using the proposed deep learning approach. Finally, the real-world application are analyzed and discussed.

/pdf/driver-activity-recognition-for-intelligent-vehicles-a-deep-3b5cabqg07.pdf

Driver Activity Recognition for Intelligent Vehicles: A Deep Learning Approach

Machine learning (ML) plays the core function to intellectualize the transportation systems. Recent years have witnessed the advent and prevalence of deep learning which has provoked a storm in ITS (Intelligent Transportation Systems). Consequently, traditional ML models in many applications have been replaced by the new learning techniques and the landscape of ITS is being reshaped. Under such perspective, we provide a comprehensive survey that focuses on the utilization of deep learning models to enhance the intelligence level of transportation systems. By organizing multiple dozens of relevant works that were originally scattered here and there, this survey attempts to provide a clear picture of how various deep learning models have been applied in multiple transportation applications.

Enhancing transportation systems via deep learning: A survey

Accurate short-time traffic flow prediction has gained gradually increasing importance for traffic plan and management with the deployment of intelligent transportation systems (ITSs). However, the existing approaches for short-term traffic flow prediction are unable to efficiently capture the complex nonlinearity of traffic flow, which provide unsatisfactory prediction accuracy. In this paper, we propose a deep learning based model which uses hybrid and multiple-layer architectures to automatically extract inherent features of traffic flow data. Firstly, built on the convolutional neural network (CNN) and the long short-term memory (LSTM) network, we develop an attention-based Conv-LSTM module to extract the spatial and short-term temporal features. The attention mechanism is properly designed to distinguish the importance of flow sequences at different times by automatically assigning different weights. Secondly, to further explore long-term temporal features, we propose a bidirectional LSTM (Bi-LSTM) module to extract daily and weekly periodic features so as to capture variance tendency of the traffic flow from both previous and posterior directions. Finally, extensive experimental results are presented to show that the proposed model combining the attention Conv-LSTM and Bi-LSTM achieves better prediction performance compared with other existing approaches.

A Hybrid Deep Learning Model With Attention-Based Conv-LSTM Networks for Short-Term Traffic Flow Prediction

The traffic flow prediction is becoming increasingly crucial in Intelligent Transportation Systems. Accurate prediction result is the precondition of traffic guidance, management, and control. To improve the prediction accuracy, a spatiotemporal traffic flow prediction method is proposed combined with k-nearest neighbor (KNN) and long short-term memory network (LSTM), which is called KNN-LSTM model in this paper. KNN is used to select mostly related neighboring stations with the test station and capture spatial features of traffic flow. LSTM is utilized to mine temporal variability of traffic flow, and a two-layer LSTM network is applied to predict traffic flow respectively in selected stations. The final prediction results are obtained by result-level fusion with rank-exponent weighting method. The prediction performance is evaluated with real-time traffic flow data provided by the Transportation Research Data Lab (TDRL) at the University of Minnesota Duluth (UMD) Data Center. Experimental results indicate that the proposed model can achieve a better performance compared with well-known prediction models including autoregressive integrated moving average (ARIMA), support vector regression (SVR), wavelet neural network (WNN), deep belief networks combined with support vector regression (DBN-SVR), and LSTM models, and the proposed model can achieve on average 12.59% accuracy improvement.

/pdf/spatiotemporal-traffic-flow-prediction-with-knn-and-lstm-3snxyuv8jb.pdf

Spatiotemporal Traffic Flow Prediction with KNN and LSTM

Intelligent vehicles and advanced driver assistance systems (ADAS) need to have proper awareness of the traffic context, as well as the driver status since ADAS share the vehicle control authorities with the human driver. This paper provides an overview of the ego-vehicle driver intention inference (DII), which mainly focuses on the lane change intention on highways. First, a human intention mechanism is discussed in the beginning to gain an overall understanding of the driver intention. Next, the ego-vehicle driver intention is classified into different categories based on various criteria. A complete DII system can be separated into different modules, which consist of traffic context awareness, driver states monitoring, and the vehicle dynamic measurement module. The relationship between these modules and the corresponding impacts on the DII are analyzed. Then, the lane change intention inference system is reviewed from the perspective of input signals, algorithms, and evaluation. Finally, future concerns and emerging trends in this area are highlighted.

/pdf/driver-lane-change-intention-inference-for-intelligent-4wcx2tetfv.pdf

Driver Lane Change Intention Inference for Intelligent Vehicles: Framework, Survey, and Challenges

Transportation systems might be heavily affected by factors such as accidents and weather. Specifically, inclement weather conditions may have a drastic impact on travel time and traffic flow. This study has two objectives: first, to investigate a correlation between weather parameters and traffic flow and, second, to improve traffic flow prediction by proposing a novel holistic architecture. It incorporates deep belief networks for traffic and weather prediction and decision-level data fusion scheme to enhance prediction accuracy using weather conditions. The experimental results, using traffic and weather data originated from the San Francisco Bay Area of California, corroborate the effectiveness of the proposed approach compared with the state of the art.

Improving Traffic Flow Prediction with Weather Information in Connected Cars: A Deep Learning Approach

Driving in busy highways and roads is becoming complex and challenging, as more cars are hitting the roads. Safe driving requires attentive drivers, quality perception of the environment, awareness of the situation, and critical decision making to react properly in emergency situations. This paper provides an overview on driver safety monitoring systems. We study various driver sources of inattention while providing a comprehensive taxonomy. Then, different safety systems that tackle driver inattention are reported. Furthermore, we present the new generation of driver monitoring systems within the context of Internet of Cars. Thus, we introduce the concept of integrated safety, where smart cars collect information from the driver, the car, the road, and, most importantly, the surrounding cars to build an efficient environment for the driver. We conclude by highlighting issues and emerging trends envisioned by the research community.

Recent Trends in Driver Safety Monitoring Systems: State of the Art and Challenges

In this paper, an end-to-end deep learning solution for driver distraction recognition is presented. In the proposed framework, the features from pre-trained convolutional neural networks VGG-19 are extracted. Despite the variation in illumination conditions, camera position, driver’s ethnicity, and genders in our dataset, our best fine-tuned model, VGG-19 has achieved the highest test accuracy of 95% and an average accuracy of 80% per class. The model is tested with leave-one-driver-out cross validation method to ensure generalization. The results show that our framework avoided the overfitting problem which typically occurs in low-variance datasets. A comparison between our framework with the state-of-the-art XGboost shows that the proposed approach outperforms XGBoost in accuracy by approximately 7%.

End-to-End Deep Learning for Driver Distraction Recognition

This work addresses short-term traffic flow prediction by proposing a big-data-based framework. The proposed framework uses data fusion to deal with heterogeneous data generated from various sources. The data are categorized into two types: streams of data and event-based data. In this work, Deep Belief Networks (DBNs) are used to independently predict traffic flow using streams of data, i.e., historical traffic flow and weather data, and event-based data, i.e., tweets. Furthermore, Dempster's conditional rule for updating belief is used to fuse evidence coming from streams of data and event-based data modules to achieve enhanced prediction. The experimental results using real-world data show the merit of the proposed framework compared to the state-of-the-art ones.

Big-data-generated traffic flow prediction using deep learning and dempster-shafer theory

In this paper, methods for improved design of digital tracking controller for a pilot-scale unmanned helicopter are considered. By discretizing the linearized helicopter model, the linear quadratic with integral (LQI) capability is investigated and applied in order to develop an efficient tracking system including a state-feedback plus integral action. The helicopter velocities are used to formulate a prescribed position reference tracking trajectory. When both process and measurement noises are present, a Kalman filter (KF) is combined with the LQI to form a linear quadratic Gaussian with integral (LQGI) tracking system. Simulation studies illuminate both the capability of the controller design and the accuracy of the estimator. Next, H2, H ∞ and mixed H 2 / H ∞ controls are designed and the results between methods are produced and compared.

Arief Koesdwiady

Papers

Improving Traffic Flow Prediction with Weather Information in Connected Cars: A Deep Learning Approach

Recent Trends in Driver Safety Monitoring Systems: State of the Art and Challenges

End-to-End Deep Learning for Driver Distraction Recognition

Big-data-generated traffic flow prediction using deep learning and dempster-shafer theory

Improved digital tracking controller design for pilot-scale unmanned helicopter