Journal ArticleDOI
Deep Reinforcement Learning-Based Controller for SOC Management of Multi-Electrical Energy Storage System
Reads0
Chats0
TLDR
A deep reinforcement learning (DRL) based controller to manage the state of charge (SOC) of a Multi-EESS, providing frequency response services to the power grid, is proposed and results show the effectiveness of the proposed approach.Abstract:
The ongoing reduction of the total rotational inertia in modern power systems brings about faster frequency dynamics that must be limited to maintain a secure and economical operation. Electrical energy storage systems (EESSs) have become increasingly attractive to provide fast frequency response services due to their response times. However, proper management of their finite energy reserves is required to ensure timely and secure operation. This paper proposes a deep reinforcement learning (DRL) based controller to manage the state of charge (SOC) of a Multi-EESS (M-EESS), providing frequency response services to the power grid. The proposed DRL agent is trained using an actor-critic method called Deep Deterministic Policy Gradients (DDPG) that allows for continuous action and smoother SOC control of the M-EESS. Deep neural networks (DNNs) are used to represent the actor and critic policies. The proposed strategy comprises granting the agent a constant reward for each time step that the SOC is within a specific band of its target value combined with a substantial penalty if the SOC reaches its minimum or maximum allowable values. The proposed controller is compared to benchmark DRL methods and other control techniques, i.e., Fuzzy Logic and a traditional PID control. Simulation results show the effectiveness of the proposed approach.read more
Citations
More filters
Journal ArticleDOI
Reinforcement Learning for Selective Key Applications in Power Systems: Recent Advances and Future Challenges
TL;DR: In this paper , a comprehensive review of various RL techniques and how they can be applied to decision-making and control in power systems is presented, including frequency regulation, voltage control, and energy management.
Journal ArticleDOI
Multi-agent deep reinforcement learning for resilience-driven routing and scheduling of mobile energy storage systems
Yi Wang,Dawei Qiu,Goran Strbac +2 more
TL;DR: In this paper , a model-free real-time multi-agent deep reinforcement learning approach featuring parameterized double deep Q-networks is proposed to reformulate the coordination effect of MESSs routing and scheduling process as a partially observable Markov game, which is capable of capturing a hybrid policy including both discrete and continuous actions.
Journal ArticleDOI
Prosumer in smart grids based on intelligent edge computing: A review on Artificial Intelligence Scheduling Techniques
TL;DR: In this article, a comprehensive review of both emerging issues and edge computing in the smart grid environment is discussed and explained, and two primary components to the energy sharing process among Prosumers: information/digital technologies and Artificial Intelligence Scheduling Techniques.
Journal ArticleDOI
An extended Kalman filter based SOC estimation method for Li-ion battery
TL;DR: Li et al. as discussed by the authors applied Thevenin equivalent circuit model of a battery to establish estimation model, and it can reflect the working state of the battery, and the extended Kalman filtering algorithm is employed to solve the estimation error caused by Gaussian noise.
Journal ArticleDOI
A Review of Second-Life Lithium-Ion Batteries for Stationary Energy Storage Applications
TL;DR: In this article , the research progress of second-life lithium-ion batteries for stationary energy storage applications, including battery aging mechanisms, repurposing, modeling, battery management, and optimal sizing, is reviewed.
References
More filters
Book
Reinforcement Learning: An Introduction
TL;DR: This book provides a clear and simple account of the key ideas and algorithms of reinforcement learning, which ranges from the history of the field's intellectual foundations to the most recent developments and applications.
Journal ArticleDOI
Approximation by superpositions of a sigmoidal function
TL;DR: It is demonstrated that finite linear combinations of compositions of a fixed, univariate function and a set of affine functionals can uniformly approximate any continuous function ofn real variables with support in the unit hypercube.
Posted Content
Playing Atari with Deep Reinforcement Learning
Volodymyr Mnih,Koray Kavukcuoglu,David Silver,Alex Graves,Ioannis Antonoglou,Daan Wierstra,Martin Riedmiller +6 more
TL;DR: This work presents the first deep learning model to successfully learn control policies directly from high-dimensional sensory input using reinforcement learning, which outperforms all previous approaches on six of the games and surpasses a human expert on three of them.
Deep reinforcement learning with double Q-learning
TL;DR: In this article, the authors show that the DQN algorithm suffers from substantial overestimation in some games in the Atari 2600 domain, and they propose a specific adaptation to the algorithm and show that this algorithm not only reduces the observed overestimations, but also leads to much better performance on several games.
Posted Content
Continuous control with deep reinforcement learning
Timothy P. Lillicrap,Jonathan J. Hunt,Alexander Pritzel,Nicolas Heess,Tom Erez,Yuval Tassa,David Silver,Daan Wierstra +7 more
TL;DR: This work presents an actor-critic, model-free algorithm based on the deterministic policy gradient that can operate over continuous action spaces, and demonstrates that for many of the tasks the algorithm can learn policies end-to-end: directly from raw pixel inputs.