scispace - formally typeset
Search or ask a question
Journal ArticleDOI

Q-Learning based Maximum Power Extraction for Wind Energy Conversion System With Variable Wind Speed

TL;DR: This paper presents an intelligent wind speed sensor less maximum power point tracking method for a variable speed wind energy conversion system (VS-WECS) based on a Q-Learning algorithm which is equipped with peak detection technique, which drives the system towards peak power even if learning is incomplete which makes the real time tracking faster.
Abstract: This paper presents an intelligent wind speed sensor less maximum power point tracking (MPPT) method for a variable speed wind energy conversion system (VS-WECS) based on a Q-Learning algorithm. The Q-Learning algorithm consists of Q-values for each state action pair which is updated using reward and learning rate. Inputs to define these states are electrical power received by grid and rotational speed of the generator. In this paper, Q-Learning is equipped with peak detection technique, which drives the system towards peak power even if learning is incomplete which makes the real time tracking faster. To make the learning uniform, each state has its separate learning parameter instead of common learning parameter for all states as is the case in conventional Q-Learning. Therefore, if half learned system is running at peak point, it does not affect the learning of unvisited states. Also, wind speed change detection is combined with proposed algorithm which makes it eligible to work for varying wind speed conditions. In addition, the information of wind turbine characteristics and wind speed measurement is not needed. The algorithm is verified through simulations and experimentation and also compared with perturbation and observation (P&O) algorithm.
Citations
More filters
Journal ArticleDOI
TL;DR: This paper proposes four improved RF methods that aim to reduce at first the amount of the training data and select the first kernel principal components using different kernel principal component analysis (PCA) based dimensionality reduction schemes.
Abstract: Random Forest (RF) is one of the mostly used machine learning techniques in fault detection and diagnosis of industrial systems. However, its implementation suffers from certain drawbacks when considering the correlations between variables. In addition, to perform a fault detection and diagnosis, the classical RF only uses the raw data by the direct use of measured variables. The direct raw data could yield to poor performance due to the data redundancies and noises. Thus, this paper proposes four improved RF methods to overcome the above-mentioned limitations. The developed methods aim to reduce at first the amount of the training data and select the first kernel principal components (KPCs) using different kernel principal component analysis (PCA) based dimensionality reduction schemes. Then, the retained KPCs are fed to the RF classifier for fault diagnosis purposes. Finally, the proposed techniques are applied to a wind energy conversion (WEC) system. Different case studies were investigated in order to illustrate the effectiveness and robustness of the developed techniques compared to the state-of-the-art methods. The obtained results show the low computation time and high diagnosis accuracy of the proposed approaches (an average accuracy of 91%).

45 citations


Cites background from "Q-Learning based Maximum Power Extr..."

  • ...Thus, the manufacturers are persistently working on improving the reliability of WTs which leads to a longevity increase, a reduction in breakdowns during the operation, and thereby an increase of the total electrical energy production [4], [5]....

    [...]

Journal ArticleDOI
TL;DR: In this paper , a comprehensive review of various RL techniques and how they can be applied to decision-making and control in power systems is presented, including frequency regulation, voltage control, and energy management.
Abstract: With large-scale integration of renewable generation and distributed energy resources, modern power systems are confronted with new operational challenges, such as growing complexity, increasing uncertainty, and aggravating volatility. Meanwhile, more and more data are becoming available owing to the widespread deployment of smart meters, smart sensors, and upgraded communication networks. As a result, data-driven control techniques, especially reinforcement learning (RL), have attracted surging attention in recent years. This paper provides a comprehensive review of various RL techniques and how they can be applied to decision-making and control in power systems. In particular, we select three key applications, i.e., frequency regulation, voltage control, and energy management, as examples to illustrate RL-based models and solutions. We then present the critical issues in the application of RL, i.e., safety, robustness, scalability, and data. Several potential future directions are discussed as well.

41 citations

Journal ArticleDOI
03 Jan 2022-Energies
TL;DR: In this article , a microgrid consisting of renewable sources, Li-ion batteries, the main grid as a backup system, and AC/DC loads was designed for effective microgrid operations using three smart controllers.
Abstract: Microgrids, comprising distributed generation, energy storage systems, and loads, have recently piqued users’ interest as a potentially viable renewable energy solution for combating climate change. According to the upstream electricity grid conditions, microgrid can operate in grid-connected and islanded modes. Energy storage systems play a critical role in maintaining the frequency and voltage stability of an islanded microgrid. As a result, several energy management systems techniques have been proposed. This paper introduces a microgrid system, an overview of local control in a microgrid, and an efficient EMS for effective microgrid operations using three smart controllers for optimal microgrid stability. We designed a microgrid consisting of renewable sources, Li-ion batteries, the main grid as a backup system, and AC/DC loads. The proposed system control was based on supplying loads as efficiently as possible using renewable energy sources and monitoring the battery’s state of charge. The simulation results using MATLAB Simulink demonstrate the performance of the three proposed microgrid stability strategies (PID, artificial neural network, and fuzzy logic). The comparison results confirmed the viability and effectiveness of the proposed technique for energy management in a microgrid which is based on fuzzy logic controllers.

18 citations

Journal ArticleDOI
TL;DR: In this article, a cascade-forward neural network (CFNN) was used to learn the wind turbine's aerodynamic nonlinear dynamics and achieves accurate power tracking, and then it reformulates the machine d-q axes voltages equations to operate the wind energy conversion systems (WECS) in optimal condition by considering the wind speed, air temperature, power demand, and disturbances.
Abstract: The demand for wind turbines has been ultimately increased over the last decades. Accordingly, the power converter controller plays the primary role in extracting energy out of the generator, using efficient and reliable techniques as Maximum Power Extraction (MPE) and delivering the power to the grid. This research pursues to present a Cascade-Forward Neural Network (CFNN) MPE that maintains the MPE's advantages besides providing the flexibility of limiting the output power at significantly lower complexity in the control loop. The proposed strategy uses the cascade-forward neural network to learn the wind turbine's aerodynamic nonlinear dynamics and achieves accurate power tracking. Additionally, it reformulates the machine d-q axes voltages equations to operate the wind energy conversion systems (WECS) in optimal condition by considering the wind speed, air temperature, power demand, and disturbances. Furthermore, it does not require any tuning procedure. The power tracking performance of the recommended CFNN MPE controller is evaluated through several experimental and simulation tests in different situations, and all the results are matched with the manufacturer's datasheets and another proven strategy to confirm its effectiveness.

15 citations

Journal ArticleDOI
TL;DR: The Q-learning algorithm is used to do phase compensation in the field of CBC to show that the Q- learning algorithm is easier to debug and has better stability.

14 citations

References
More filters
Book
01 Jan 1988
TL;DR: This book provides a clear and simple account of the key ideas and algorithms of reinforcement learning, which ranges from the history of the field's intellectual foundations to the most recent developments and applications.
Abstract: Reinforcement learning, one of the most active research areas in artificial intelligence, is a computational approach to learning whereby an agent tries to maximize the total amount of reward it receives when interacting with a complex, uncertain environment. In Reinforcement Learning, Richard Sutton and Andrew Barto provide a clear and simple account of the key ideas and algorithms of reinforcement learning. Their discussion ranges from the history of the field's intellectual foundations to the most recent developments and applications. The only necessary mathematical background is familiarity with elementary concepts of probability. The book is divided into three parts. Part I defines the reinforcement learning problem in terms of Markov decision processes. Part II provides basic solution methods: dynamic programming, Monte Carlo methods, and temporal-difference learning. Part III presents a unified view of the solution methods and incorporates artificial neural networks, eligibility traces, and planning; the two final chapters present case studies and consider the future of reinforcement learning.

37,989 citations


"Q-Learning based Maximum Power Extr..." refers background in this paper

  • ...In the reinforcement learning, the focus is on direct interaction of the individual (agent) with its environment, which learns from its own experience [17]....

    [...]

01 Jan 1989

4,916 citations

Journal ArticleDOI
TL;DR: The invention relates to a circuit for use in a receiver which can receive two-tone/stereo signals which is intended to make a choice between mono or stereo reproduction of signal A or of signal B and vice versa.

2,861 citations


"Q-Learning based Maximum Power Extr..." refers methods in this paper

  • ...been made by Watkins [18] who has suggested new algorithm called Q-learning and applied it to MDP....

    [...]

  • ...For optimum policy, selection of action is done such that the maximum discounted reward can be obtained for each state. π∗ (s) = arg max a ∑ s′ P (s, a, s′)V ∗ (s′) (6) For a transition from state s to state s’, the update for value is as, V π (s) = V π (s) + η (r (s) + Υ V π (s′)− V π (s)) (7) Most important breakthrough in reinforcement learning has been made by Watkins [18] who has suggested new algorithm called Q-learning and applied it to MDP. Q-learning is the first reinforcement learning algorithm whose convergence to optimal policy is proven for decision making problems involving cumulative cost....

    [...]

  • ...[18] C. J. C. H. Watkins, “Learning from delayed rewards,” Ph.D. Dissertation, Dept. Psychol., Cambridge Univ., Cambridge, England, 1989....

    [...]

Journal ArticleDOI
TL;DR: In this paper, an intelligent maximum power extraction algorithm is developed by the authors to improve the system performance and to facilitate the control implementation, where an advanced hill-climb searching method is developed to take into account the wind turbine inertia.
Abstract: This paper focuses on the development of maximum wind power extraction algorithms for inverter-based variable speed wind power generation systems. A review of existing maximum wind power extraction algorithms is presented in this paper, based on which an intelligent maximum power extraction algorithm is developed by the authors to improve the system performance and to facilitate the control implementation. As an integral part of the max-power extraction algorithm, advanced hill-climb searching method has been developed to take into account the wind turbine inertia. The intelligent memory method with an on-line training process is described in this paper. The developed maximum wind power extraction algorithm has the capability of providing initial power demand based on error driven control, searching for the maximum wind turbine power at variable wind speeds, constructing an intelligent memory, and applying the intelligent memory data to control the inverter for maximum wind power extraction, without the need for either knowledge of wind turbine characteristics or the measurements of mechanical quantities such as wind speed and turbine rotor speed. System simulation results and test results have confirmed the functionality and performance of this method.

507 citations

Journal ArticleDOI
TL;DR: A novel peak detection capability has been devised which, in contrast with conventional peak detection, can work robustly under changing wind conditions and performs self-tuning to cope with the nonconstant efficiencies of the generator-converter subsystems.
Abstract: This paper proposes a novel solution to the problems that exist in the conventional hill climb searching (HCS) maximum power point tracking (MPPT) algorithm for the wind energy conversion system. The presented solution not only solves the tracking speed versus control efficiency tradeoff problem of HCS but also makes sure that the changing wind conditions do not lead HCS in the wrong direction. It intelligently adapts the variable step size to keep up with the rapid changes in the wind and seizes the perturbation at the maxima to yield 100% control efficiency. For this purpose, a novel peak detection capability has been devised which, in contrast with conventional peak detection, can work robustly under changing wind conditions. The proposed MPPT performs self-tuning to cope with the nonconstant efficiencies of the generator-converter subsystems-a phenomenon quite rarely discussed in research papers so far. In addition, a smart speed-sensorless scheme has been developed to avoid the use of mechanical sensors. The experimental results confirm that the proposed algorithm is remarkably faster and more efficient than the conventional HCS.

408 citations