Ming-Li Zhang

Bio: Ming-Li Zhang is an academic researcher from Yanshan University. The author has contributed to research in topics: Artificial neural network & Deep learning. The author has an hindex of 1, co-authored 2 publications receiving 3 citations.

An Improved DDPG and Its Application Based on the Double-Layer BP Neural Network

Abstract: This paper focused on three application problems of the traditional Deep Deterministic Policy Gradient(DDPG) algorithm. That is, the agent exploration is insufficient, the neural network performance is unsatisfied, the agent output fluctuates greatly. In terms of agent exploration strategy, network training algorithm and overall algorithm implementation, an improved DDPG method based on double-layer BP neural network is proposed. This method introduces fuzzy algorithm and BFGS algorithm based on Armijo-Goldstein criterion, improves the exploration efficiency, learning efficiency and convergence of BP neural network, increases the number of layers of BP neural network to improve the fitting ability of the network, and adopts periodic update to ensure the stable operation of the algorithm. The experimental results show that the deep learning network based on the improved DDPG algorithm has greatly improved the performance compared with the traditional method after multiple rounds of self-learning under variable working conditions. This study lays a theoretical and experimental foundation for the extended application of deep learning algorithm.

Adaptive PID Control and Its Application Based on a Double-Layer BP Neural Network

Abstract: In this paper, focusing on the inconvenience of variable value PID based on manual parameter adjustment for the hydraulic drive unit (HDU) of a legged robot, a method employing double-layer back propagation (BP) neural networks for learning the law of PID control parameters is proposed. The first layer is used to learn the relationship between different control parameters and the control performance of the system under various working conditions. The second layer is used to study the relationship between the parameters of the working conditions and the optimizing control parameters under various working conditions. The effectiveness of the proposed control method was verified by simulation and experiment. The results showed that the proposed method can provide a theoretical and experimental basis for the selection of control parameters, and can be extended to similar controllers, therefore possessing engineering application value.

Spatio-Temporal Characteristics of the Supply and Demand Coupling Coordination of Elderly Care Service Resources in China

Abstract: The current situation and future development of the supply and demand coupling coordination of elderly care service resources reflect the level of elderly care service resource allocation. Whether factors affecting its development can be found is the key to promote the accurate allocation of elderly care service. Based on the coupling coordination model, the supply and demand of elderly care service resources, the development circumstance and the spatio-temporal evolution of supply and demand coupling coordination are analyzed in this paper by using the data of the elderly care service resources in 31 regions and autonomous regions in China from 2010 to 2019. The result shows that there are regional differences in the development of supply and demand coupling coordination of elderly care service resources. The degree of supply and demand coupling coordination of elderly care service resources in the western and northern regions is lower than that in the eastern and southern regions. Although the level in most areas of supply and demand coupling coordination of elderly care service resources will improve in the future, there is still a gap from good coordination. In order to strengthen the supply of elderly care service resources, and promote the upgrade of the supply and demand of elderly care service resources, the government should start from the demand of the elderly to increase investment in infrastructure construction, investment in elderly care services resources, talent training and other aspects.

Path Planning Based on Deep Reinforcement Learning for Autonomous Underwater Vehicles Under Ocean Current Disturbance

Abstract: The path planning issue of the underactuated autonomous underwater vehicle (AUV) under ocean current disturbance is studied in this paper. In order to improve the AUV’s path planning capability in the unknown environments, a deep reinforcement learning (DRL) path planning method based on double deep Q Network (DDQN) is proposed. It is created from an improved convolutional neural network, which has two input layers to adapt to the processing of high-dimensional environments. Considering the maneuverability of underactuated AUV under current disturbance, especially, the issue of ocean current disturbance under unknown environments, a dynamic and composite reward function is developed to enable the AUV to reach the destination with obstacle avoidance. Finally, the path planning ability of the proposed method in the unknown environments is validated by simulation analysis and comparison studies.

Data‐driven optimal scheduling for underground space based integrated hydrogen energy system

Abstract: Integrated hydrogen energy systems (IHESs) have attracted extensive attention in mitigating climate problems. As a kind of large-scale hydrogen storage device, underground hydrogen storage (UHS) can be introduced into IHES to balance the seasonal energy mismatch, while bringing challenges to optimal operation of IHES due to the complex geological structure and uncertain hydrodynamics. To address this problem, a deep deterministic policy gradient (DDPG)-based optimal scheduling method for underground space based IHES is proposed. The energy management problem is formulated as a Markov decision process to characterize the interaction between environmental states and policy. Based on DDPG theory, the actor-critic structure is applied to approximate deterministic policy and actor-value function. Through policy iteration and actor-critic network training, the operation of UHS and other energy conversion devices can be adaptively optimised, which is driven by real-time response data instead of accurate system models. Finally, the effectiveness of the proposed optimal scheduling method and the benefits of underground space are verified through time-domain simulations.

Path Planning Based on Deep Reinforcement Learning for Autonomous Underwater Vehicles Under Ocean Current Disturbance

Abstract: The path planning issue of the underactuated autonomous underwater vehicle (AUV) under ocean current disturbance is studied in this paper. In order to improve the AUV’s path planning capability in the unknown environments, a deep reinforcement learning (DRL) path planning method based on double deep Q Network (DDQN) is proposed. It is created from an improved convolutional neural network, which has two input layers to adapt to the processing of high-dimensional environments. Considering the maneuverability of underactuated AUV under current disturbance, especially, the issue of ocean current disturbance under unknown environments, a dynamic and composite reward function is developed to enable the AUV to reach the destination with obstacle avoidance. Finally, the path planning ability of the proposed method in the unknown environments is validated by simulation analysis and comparison studies.

Machine Learning Approach Based on Ultra-Local Model Control for Treating Cancer Pain

Abstract: Cancer illness still is one of the most common illnesses in the world, which is constantly rising. Chemotherapy plays a crucial role in treating cancer patients. In this paper, we have presented a novel intelligent sensor for controlling and adjusting chemotherapy parameters which consist of an ultra-local (ULM) controller based on a deep deterministic policy gradient (DDPG). First, the feedback signal is provided using a sensor to calculate the population of cells. Then, a controller sends the proper control commands to the actuator (chemotherapy). In the suggested scheme, the ULM is applied to the dynamic model of cancer. In order to shrink tumor cells and rising immune and normal cells at the same time. Moreover, for improving the performance of the established ULM scheme, a DDPG algorithm with the actor-critic structure is used for tuning the parameters of ULM in an adaptive manner. To demonstrate the supremacy of the DDPG based ULM controller, the conventional ULM and proportional integrator (PI) are also designed for the cancer treatment. Simulation outcomes prove the improved cancer treatment compared to the ULM and PI schemes.