Rough pavements cause ride discomfort and energy inefficiency for road vehicles. Existing methods to address these problems are time-consuming and not adaptive to changing driving conditions on rough pavements. With the development of sensor and communication technologies, crowdsourced road and dynamic traffic information become available for enhancing driving performance, particularly addressing the discomfort and inefficiency issues by controlling driving speeds. This study proposes a speed control framework on rough pavements, envisioning the operation of autonomous vehicles based on the crowdsourced data. We suggest the concept of ‘maximum comfortable speed’ for representing the vertical ride comfort of oncoming roads. A deep reinforcement learning (DRL) algorithm is designed to learn comfortable and energy-efficient speed control strategies. The DRL-based speed control model is trained using real-world rough pavement data in Shanghai, China. The experimental results show that the vertical ride comfort, energy efficiency, and computation efficiency increase by 8.22%, 24.37%, and 94.38%, respectively, compared to an optimization-based speed control model. The results indicate that the proposed framework is effective for real-time speed controls of autonomous vehicles on rough pavements. 

Comfortable and energy-efficient speed control of autonomous vehicles on rough pavements using deep reinforcement learning

Several studies exist on topics of semi-active suspension and vehicle cruise control systems in the literature, while many of them just consider actual road distortions and terrain characteristics, these systems are not adaptive and their subsystems designed separately. This study introduces a new method where the integration of look-ahead road data in the control of the adaptive semi-active suspension, where it is possible to the trade-off between comfort and stability orientation. This trade-off is designed by the decision layer, where the controller is modified based on prehistorical passive suspension simulations, vehicle velocity and road data, while the behavior of the controller can be modified by the use of a dedicated scheduling variable. The adaptive semi-active suspension control is designed by using Linear Parameter Varying (LPV) framework. In addition to this, it proposes designing the vehicle velocity for the cruise controller by considering energy efficiency and comfort together. TruckSim environment is used to validate the operation of the proposed integrated cruise and semi-active suspension control system.

Adaptive Semi-Active Suspension and Cruise Control through LPV Technique

A Survey on Deep Reinforcement Learning Libraries

Design of tracking controller for quadrotor is an important issue for many engineering fields such as COVID-19 epidemic prevention, intelligent agriculture, military photography and rescue nowadays. This study applies the feedback linearized method and linear quadratic regulator (LQR) method using particle swarm optimization (PSO) to analysis and stabilize the highly nonlinear quadrotor system without applying any nonlinear function approximator that includes neural network approach and fuzzy approach. The article proposes a new method based on the firstly proposed convergence rate formula to achieve the optimal weighting matrices of LQR such that the composite controller can reduce the amplitudes of system control inputs. Determination of the LQR tuning parameters is conventionally achieved via trial and error approach. In addition to being very troublesome, it is difficult to find the globally best tuning matrices with LQR method. This article firstly uses the convergence rate formula of the nonlinear system as the fitness function of LQR approach by using PSO to take the place of the trial and error method. The generalities and implications of proposed approach are globally valid, whereas the Jacobian linearized approach is locally valid due to the Taylor expansion theorem. In addition to these two major achievements, the significant innovation of the proposed method is to possess “simultaneously” additional performances including the almost disturbance decoupling, input amplitude reduction, tuning parameter optimization and globally exponential stability performances. Comparative examples show that the convergence rate with our proposed optimal controller using the PSO algorithm is larger than the fuzzy method, and better than the singular perturbation method with high-gain feedback.

/pdf/feedback-linearized-optimal-control-design-for-quadrotor-1fekxdmf3r.pdf

Feedback Linearized Optimal Control Design for Quadrotor With Multi-Performances

Ride comfort plays an important role in determining the public acceptance of autonomous vehicles (AVs). Many factors, such as road profile, driving speed, and suspension system, influence the ride comfort of AVs. This study proposes a hierarchical framework for improving ride comfort by integrating speed planning and suspension control in a vehicle‐to‐everything environment. Based on safe, comfortable, and efficient speed planning via dynamic programming, a deep reinforcement learning‐based suspension control is proposed to adapt to the changing pavement conditions. Specifically, a deep deterministic policy gradient with external knowledge (EK‐DDPG) algorithm is designed for the efficient self‐adaptation of suspension control strategies. The external knowledge of action selection and value estimation from other AVs are combined into the loss functions of the DDPG algorithm. In numerical experiments, real‐world pavements detected in 11 districts of Shanghai, China, are applied to verify the proposed method. Experimental results demonstrate that the EK‐DDPG‐based suspension control improves ride comfort on untrained rough pavements by 27.95% and 3.32%, compared to a model predictive control (MPC) baseline and a DDPG baseline, respectively. Meanwhile, the EK‐DDPG‐based suspension control improves computational efficiency by 22.97%, compared to the MPC baseline, and performs at the same level as the DDPD baseline. This study provides a generalized and computationally efficient approach for improving the ride comfort of AVs.

A hierarchical framework for improving ride comfort of autonomous vehicles via deep reinforcement learning with external knowledge

The performance of vehicle body vibration and ride comfort of active or semi-active suspension with proper control is better than that with passive suspension. The key to achieve good control effect is that the suspension control system should have strong real-time learning ability according to changes in the road surface and suspension parameters. In the control strategies adopted by previous researchers, the classical neural network controller has some learning ability, but it is mainly based on offline learning with a large number of samples. In this paper, the deep reinforcement learning strategy is used to solve the above problems.Aiming at the continuity of state space and execution action in vehicle active suspension system, the control of the semi-active suspension is realized by using improved DDPG (Deep Deterministic Policy Gradient) algorithm. To overcome the shortcoming of low efficiency of this algorithm in the initial stage of learning, the DDPG algorithm is improved and using empirical samples in the learning method is proposed. Based on Mujoco, the physical model of semi-active suspension is established, and its dynamic characteristics are analyzed under the condition of various road level and vehicle speed. The simulation results show that compared with the passive suspension, the semi-active suspension based on improved DDPG algorithm with learning method using experienced samples can better adapt to various road level, more effectively reduce the vertical acceleration of the vehicle body and the dynamic deflection of the suspension, and further improve the ride comfort.

/pdf/semi-active-suspension-control-based-on-deep-reinforcement-3elm5mcmcm.pdf

Yin Yan-fang

Papers

Semi-Active Suspension Control Based on Deep Reinforcement Learning