https://dc.tsinghuajournals.com/cgi/viewcontent.cgi?article=1565&context=tsinghua-science-and-technology

Deep reinforcement learning based mobile robot navigation: A review

This paper comprehensively reviews recent developments in modeling lane-changing behavior. The major lane changing models in the literature are categorized into two groups: models that aim to capture the lane changing decision-making process, and models that aim to quantify the impact of lane changing behavior on surrounding vehicles. The methodologies and important features (including their limitations) of representative models in each category are outlined and discussed. Future research needs are determined.

Recent developments and research needs in modeling lane changing

The challenge and opportunity of battery lifetime prediction from field data

Driving behavior has a large impact on vehicle fuel consumption. Dedicated study on the relationship between the driving behavior and fuel consumption can contribute to decreasing the energy cost of transportation and the development of the behavior assessment technology for the ADAS system. Therefore, it is vital to evaluate this relationship in order to develop more ecological driving assistance systems and improve the vehicle fuel economy. However, modeling driving behavior under the dynamic driving conditions is complex, making a quantitative analysis of the relationship between the driving behavior and the fuel consumption difficult. In this paper, we introduce two kinds of machine learning methods for evaluating the fuel efficiency of driving behavior using the naturalistic driving data. In the first stage, we use an unsupervised spectral clustering algorithm to study the macroscopic relationship between driving behavior and fuel consumption, using the data collected during the natural driving process. In the second stage, the dynamic information from the driving environment and natural driving data is integrated to generate a model of the relationship between various driving behaviors and the corresponding fuel consumption features. The dynamic environment factors are coded into a processable, digital form using a deep learning-based object detection method so that the environmental data can be linked with the vehicle's operating signal data to provide the training data for the deep learning network. The training data are labeled according to its fuel consumption feature distribution, which is obtained from the road segment data and historical driving data. This deep learning-based model can then be used as a predictor of the fuel consumption associated with different driving behaviors. Our results show that the proposed method can effectively identify the relationship between the driving behavior and the fuel consumption on both macro and micro levels, allowing for end-to-end fuel consumption feature prediction, which can then be applied in the advanced driving assistance systems.

/pdf/impact-of-driver-behavior-on-fuel-consumption-classification-1yemebnoru.pdf

Impact of Driver Behavior on Fuel Consumption: Classification, Evaluation and Prediction Using Machine Learning

In recent days, the deployment of electric vehicles (EVs) in automobile sector is increasing the load demand in the distribution system. To deal with this load demand, the charging management needs to be improved. Nevertheless, an EV needs several hours to complete charge. Reducing the charging time, energy consumption is a huge contest to deal with the promotion of EV over conventional vehicles. The condition of the itineraries may affect the energy consumption of the EV, which needs to be considered before fulfilling the energy demand. In this article, these are considered assigning suitable charging stations (CS) to individual EVs and their scheduling is taken as an optimization problem. The first part deals with the proper assignment of CS, which is a linear optimization problem and the second deals with the charging scheduling problem. An “intelligent charging scheduling algorithm (ICSA)” is proposed with the integration of Henry gas solubility optimization to minimize total daily price incurred by the CS operator. Later, ICSA is clubbed with other standard optimization techniques considering practical constraints. A 2 m point estimation method has been utilized to tackle the uncertainty and its performance has been compared with the Monte-Carlo simulation technique. The robustness of ICSA is evaluated and confirmed using the Wilcoxon signed rank test and Quade test.

Charging Scheduling of Electric Vehicle Incorporating Grid-to-Vehicle and Vehicle-to-Grid Technology Considering in Smart Grid

This paper presents an approach for solving the crowd navigation problem in an unknown and dynamic environment based on deep reinforcement learning. In our approach, we first make four leader agents learn how to reach their goals and avoid collisions with static and dynamic obstacles in an unknown environment by use of proximal policy optimization combined with Long short-term memory and a collision prediction algorithm. In the second stage, we make each leader agent arrive at a specific goal several times and record its trajectory as the guiding path so that the members in its group know how to reach their goals. We adopt the Reciprocal Velocity Obstacle algorithm to make agents not collide with others. Finally, we simulate the scenario of four groups moving towards their goals simultaneously using the Unity 3D engine. The experimental results demonstrate self-learning ability of a crowd who can reach their goals successfully in an unknown and dynamic environment.

/pdf/crowd-navigation-in-an-unknown-and-dynamic-environment-based-4ptcp3hg5j.pdf

Crowd Navigation in an Unknown and Dynamic Environment Based on Deep Reinforcement Learning

The evaluation of lithium battery performance is a complex and very important issue. Generally, manufacturers perform battery burn-in tests and evaluate the performance of lithium batteries based on capacity, internal resistance, voltage, and other parameters in the cycle. However, due to the complexity of practical applications and the difficulty of parameter measurement, it is necessary to evaluate the status of health (SOH) of lithium-ion batteries from the side. Analysis of battery charge and discharge data found that using the charge and discharge time to evaluate the health of the battery is effective and feasible, especially the time during the discharge/charge platform period, and the parameter measurement is more convenient. In this paper, three time-health indicators are constructed and analyzed in detail, and then the health of the battery is evaluated using a simple Bayesian Monte Carlo theory. The experimental results of four batteries show that the scheme is simple and convenient, and can effectively evaluate the SOH of lithium-ion batteries.

/pdf/remaining-useful-life-estimation-of-lithium-ion-batteries-1fek9z9m7i.pdf

Remaining Useful Life Estimation of Lithium-Ion Batteries Based on Optimal Time Series Health Indicator

Research on how risk is perceived by drivers is vital to driving behavior research and driving safety. As risk can be divided into subjective and objective risk, in this paper, we focus on modeling subjective risk perception by drivers using a deep learning method. Different drivers often perceive different levels of subjective risk under the same driving conditions. In addition, different driving conditions or driving events will have different effects on drivers. Based on these two risk perception features, in this paper, we first design an experiment on a city road with two lanes to assess the level of subjective risk perceived by drivers belonging to different groups. We then use a deep learning network-based method to abstract features of the driving environment. These environmental features are integrated with driver risk perception data and this information is used as training and testing data for the learning network. Finally, a long–short-term memory-based method is adopted to model the subjective risk perception of individual drivers based on traffic conditions and vehicle operation data from the driver’s vehicle. Our results show that the proposed method can effectively model the subjective risk perception behavior of drivers, allowing for end-to-end risk perception prediction in future driving assistance systems.

Modeling Driver Risk Perception on City Roads Using Deep Learning

There have been some biomechanics-based control systems that have achieved better realistic virtual human motion. Yet their abilities to adapt the changing environments are weaker than the traditional control systems with characters driven by proportional derivative actuators directly. In our method, we build a hierarchical neuromuscular virtual human (NMVH) motion control system that consists of a low-level spine reflex layer and a high-level policy control layer. The spine reflex layer uses a feedback net to map sensory information to excitations, which stimulate muscles to generate joint torques. The policy control layer includes a deep neural network, which provides a learned action policy to spine reflex layer for achieving terrain-adaptive motion skills. The particle swarm optimization algorithm is used to optimize the gain factors of the feedback net for finding out a basic policy to make the virtual human walk on the flat terrain autonomously. The proximal policy optimization algorithm is employed to train the deep neural network in policy control layer for learning how to modulate the actions to adapt to the changing terrain. The simulation results in Matlab show that virtual human can walk smoothly and better adapt to the given terrain changes. It demonstrates that our control system improves the terrain-adaptive walking skill of the neuromuscular virtual human.

/pdf/terrain-adaptive-walking-of-biped-neuromuscular-virtual-35dr1cc8ws.pdf

Wenhu Qin

Papers

Impact of Driver Behavior on Fuel Consumption: Classification, Evaluation and Prediction Using Machine Learning

Crowd Navigation in an Unknown and Dynamic Environment Based on Deep Reinforcement Learning

Remaining Useful Life Estimation of Lithium-Ion Batteries Based on Optimal Time Series Health Indicator

Modeling Driver Risk Perception on City Roads Using Deep Learning

Terrain Adaptive Walking of Biped Neuromuscular Virtual Human Using Deep Reinforcement Learning