The path planning issue of the underactuated autonomous underwater vehicle (AUV) under ocean current disturbance is studied in this paper. In order to improve the AUV’s path planning capability in the unknown environments, a deep reinforcement learning (DRL) path planning method based on double deep Q Network (DDQN) is proposed. It is created from an improved convolutional neural network, which has two input layers to adapt to the processing of high-dimensional environments. Considering the maneuverability of underactuated AUV under current disturbance, especially, the issue of ocean current disturbance under unknown environments, a dynamic and composite reward function is developed to enable the AUV to reach the destination with obstacle avoidance. Finally, the path planning ability of the proposed method in the unknown environments is validated by simulation analysis and comparison studies.

Path Planning Based on Deep Reinforcement Learning for Autonomous Underwater Vehicles Under Ocean Current Disturbance

Integrated hydrogen energy systems (IHESs) have attracted extensive attention in mitigating climate problems. As a kind of large-scale hydrogen storage device, underground hydrogen storage (UHS) can be introduced into IHES to balance the seasonal energy mismatch, while bringing challenges to optimal operation of IHES due to the complex geological structure and uncertain hydrodynamics. To address this problem, a deep deterministic policy gradient (DDPG)-based optimal scheduling method for underground space based IHES is proposed. The energy management problem is formulated as a Markov decision process to characterize the interaction between environmental states and policy. Based on DDPG theory, the actor-critic structure is applied to approximate deterministic policy and actor-value function. Through policy iteration and actor-critic network training, the operation of UHS and other energy conversion devices can be adaptively optimised, which is driven by real-time response data instead of accurate system models. Finally, the effectiveness of the proposed optimal scheduling method and the benefits of underground space are verified through time-domain simulations. 

Data‐driven optimal scheduling for underground space based integrated hydrogen energy system

The path planning issue of the underactuated autonomous underwater vehicle (AUV) under ocean current disturbance is studied in this paper. In order to improve the AUV’s path planning capability in the unknown environments, a deep reinforcement learning (DRL) path planning method based on double deep Q Network (DDQN) is proposed. It is created from an improved convolutional neural network, which has two input layers to adapt to the processing of high-dimensional environments. Considering the maneuverability of underactuated AUV under current disturbance, especially, the issue of ocean current disturbance under unknown environments, a dynamic and composite reward function is developed to enable the AUV to reach the destination with obstacle avoidance. Finally, the path planning ability of the proposed method in the unknown environments is validated by simulation analysis and comparison studies. 

Cancer illness still is one of the most common illnesses in the world, which is constantly rising. Chemotherapy plays a crucial role in treating cancer patients. In this paper, we have presented a novel intelligent sensor for controlling and adjusting chemotherapy parameters which consist of an ultra-local (ULM) controller based on a deep deterministic policy gradient (DDPG). First, the feedback signal is provided using a sensor to calculate the population of cells. Then, a controller sends the proper control commands to the actuator (chemotherapy). In the suggested scheme, the ULM is applied to the dynamic model of cancer. In order to shrink tumor cells and rising immune and normal cells at the same time. Moreover, for improving the performance of the established ULM scheme, a DDPG algorithm with the actor-critic structure is used for tuning the parameters of ULM in an adaptive manner. To demonstrate the supremacy of the DDPG based ULM controller, the conventional ULM and proportional integrator (PI) are also designed for the cancer treatment. Simulation outcomes prove the improved cancer treatment compared to the ULM and PI schemes.

Machine Learning Approach Based on Ultra-Local Model Control for Treating Cancer Pain

Deep reinforcement learning based parameter self-tuning control strategy for VSG

This paper focused on three application problems of the traditional Deep Deterministic Policy Gradient(DDPG) algorithm. That is, the agent exploration is insufficient, the neural network performance is unsatisfied, the agent output fluctuates greatly. In terms of agent exploration strategy, network training algorithm and overall algorithm implementation, an improved DDPG method based on double-layer BP neural network is proposed. This method introduces fuzzy algorithm and BFGS algorithm based on Armijo-Goldstein criterion, improves the exploration efficiency, learning efficiency and convergence of BP neural network, increases the number of layers of BP neural network to improve the fitting ability of the network, and adopts periodic update to ensure the stable operation of the algorithm. The experimental results show that the deep learning network based on the improved DDPG algorithm has greatly improved the performance compared with the traditional method after multiple rounds of self-learning under variable working conditions. This study lays a theoretical and experimental foundation for the extended application of deep learning algorithm.

/pdf/an-improved-ddpg-and-its-application-based-on-the-double-26cb603e68.pdf

An Improved DDPG and Its Application Based on the Double-Layer BP Neural Network

With the rapid growth of the elderly population of China in recent years, the service demands of older Chinese people continue to increase. The increasingly severe situation with respect to the elderly population is an important social problem that China will face for a long time into the future. It is urgent to solve the problem of how to scientifically carry out allocation planning of service resources for the aged and guide the effective supply of service resources. This paper analyzes the factors affecting service resources for the aged, divides China’s service resource supply and demand system into a supply subsystem, a demand subsystem, and a population and economy subsystem. Using system dynamics methods to analyze the causal relationship between variables and the state space method to build a mathematical model and perform simulation analysis, we research the the current situation of China’s service resources supply and demand balance for the aged. In addition, we put forward resource configuration optimization measures for the future allocation of service resources for the aged, providing a practical basis for future decision-making.

/pdf/research-on-supply-and-demand-of-aged-services-resource-ri5f83lo.pdf

Research on Supply and Demand of Aged Services Resource Allocation in China: A System Dynamics Model

In this paper, focusing on the inconvenience of variable value PID based on manual parameter adjustment for the hydraulic drive unit (HDU) of a legged robot, a method employing double-layer back propagation (BP) neural networks for learning the law of PID control parameters is proposed. The first layer is used to learn the relationship between different control parameters and the control performance of the system under various working conditions. The second layer is used to study the relationship between the parameters of the working conditions and the optimizing control parameters under various working conditions. The effectiveness of the proposed control method was verified by simulation and experiment. The results showed that the proposed method can provide a theoretical and experimental basis for the selection of control parameters, and can be extended to similar controllers, therefore possessing engineering application value.

Adaptive PID Control and Its Application Based on a Double-Layer BP Neural Network

The current situation and future development of the supply and demand coupling coordination of elderly care service resources reflect the level of elderly care service resource allocation. Whether factors affecting its development can be found is the key to promote the accurate allocation of elderly care service. Based on the coupling coordination model, the supply and demand of elderly care service resources, the development circumstance and the spatio-temporal evolution of supply and demand coupling coordination are analyzed in this paper by using the data of the elderly care service resources in 31 regions and autonomous regions in China from 2010 to 2019. The result shows that there are regional differences in the development of supply and demand coupling coordination of elderly care service resources. The degree of supply and demand coupling coordination of elderly care service resources in the western and northern regions is lower than that in the eastern and southern regions. Although the level in most areas of supply and demand coupling coordination of elderly care service resources will improve in the future, there is still a gap from good coordination. In order to strengthen the supply of elderly care service resources, and promote the upgrade of the supply and demand of elderly care service resources, the government should start from the demand of the elderly to increase investment in infrastructure construction, investment in elderly care services resources, talent training and other aspects.

/pdf/spatio-temporal-characteristics-of-the-supply-and-demand-3a8v7tcz.pdf

Spatio-Temporal Characteristics of the Supply and Demand Coupling Coordination of Elderly Care Service Resources in China

The rapid increase in the proportion of the elderly population in China has increased the demand for aged services. It is a huge challenge for the Chinese government to establish the aged service system to meet such a large demand in a short time. In its 14th Five-Year Development plan, to promote a more rational allocation of health services, the Chinese government plans to build a health service system for the elderly. Based on the aged service resource allocation model established in the previous research, this paper proposes optimization measures and improves the model. The second-order matrix sensitivity analysis is performed on 30 parameters in the improved model, and the influence law of parameters on the whole system under a different amplitude of variation is explored. The results show that the disability rate of the elderly population, the service capacity of aged service institutions, per capita disposable income, and other factors have a great impact on the balance between the supply and demand of aged service resources. Therefore, when formulating policies related to the allocation of aged service resources, we can focus on improving the health level of the elderly, increasing the investment of social funds in aged service, and improving people’s consumption levels, so as to further improve the aged service system.

/pdf/the-sensitivity-analysis-for-supply-and-demand-dynamics-150u9p2z.pdf

Yi-Jie Zhang

Papers

An Improved DDPG and Its Application Based on the Double-Layer BP Neural Network

Research on Supply and Demand of Aged Services Resource Allocation in China: A System Dynamics Model

Adaptive PID Control and Its Application Based on a Double-Layer BP Neural Network

Spatio-Temporal Characteristics of the Supply and Demand Coupling Coordination of Elderly Care Service Resources in China

The Sensitivity Analysis for Supply and Demand Dynamics System of Aged Services Resource Allocation in China