In this paper, a novel iterative $Q$ -learning method called “dual iterative $Q$ -learning algorithm” is developed to solve the optimal battery management and control problem in smart residential environments. In the developed algorithm, two iterations are introduced, which are internal and external iterations, where internal iteration minimizes the total cost of power loads in each period, and the external iteration makes the iterative $Q$ -function converge to the optimum. Based on the dual iterative $Q$ -learning algorithm, the convergence property of the iterative $Q$ -learning method for the optimal battery management and control problem is proven for the first time, which guarantees that both the iterative $Q$ -function and the iterative control law reach the optimum. Implementing the algorithm by neural networks, numerical results and comparisons are given to illustrate the performance of the developed algorithm.

A Novel Dual Iterative $Q$ -Learning Method for Optimal Battery Management in Smart Residential Environments

An optimized communication and control infrastructure, based on the microgrid building block concept, for active distribution systems is essential to facilitate powerful control framework under the smart grid paradigm. In this paper, a novel methodology for designing a communication and control infrastructure is presented. The new design takes into account both communication system and distribution system-related aspects. The proposed design facilitates systematic and optimized clustering of the distribution system into a set of virtual microgrids with optimized communication requirements while considering the power quality aspects, characteristics of distributed generation units, distributed energy storage units, and distributed reactive sources. The new design facilitates robust infrastructure for smart distribution systems operation and control, e.g., self-healing control and optimized system-level operation, by using virtual microgrids as building blocks in future distribution systems. The motivations, conceptual design, problem formulation and solution algorithms are presented in this paper. The well-known PG&E 69-bus distribution system is selected as a test case and through several sensitivity studies, the effect of optimization coefficients on the design, and the robustness of the algorithm are investigated. Finally, the algorithm is tested on the IEEE 123-bus distribution system.

Optimized Multiple Microgrid-Based Clustering of Active Distribution Systems Considering Communication and Control Requirements

Stochastic systems provide powerful abstract models for a variety of important real-life applications: for example, power supply, traffic flow, data transmission. They (and the real systems they model) are often subject to phase transitions, behaving in one way when a parameter is below a certain critical value, then switching behaviour as soon as that critical value is reached. In a real system, we do not necessarily have control over all the parameter values, so it is important to know how to find critical points and to understand system behaviour near these points. This book is a modern presentation of the 'semimartingale' or 'Lyapunov function' method applied to near-critical stochastic systems, exemplified by non-homogeneous random walks. Applications treat near-critical stochastic systems and range across modern probability theory from stochastic billiards models to interacting particle systems. Spatially non-homogeneous random walks are explored in depth, as they provide prototypical near-critical systems.

Non-homogeneous Random Walks: Lyapunov Function Methods for Near-Critical Stochastic Systems

A novel structure of Ferro-Aluminum based sandwich composite for magnetic and electromagnetic interference shielding

Intelligent energy facilities, e.g., smart grids and microgrids are the evolution of traditional energy grids through digital transformation. These modern paradigms are expected to foster the utilization of renewable energies, sustainable development, and resilience of the power grid. A barrier found when deploying experimental smart grids and microgrids consists of handling the heterogeneity of the required hardware and software components as well as the available commercial equipment. Despite the fact that there is various architecture proposed in previous literature, it commonly lacks experimental validation, specification of involved equipment concerning industrial/proprietary or open-source nature, and concretization of communication protocols. To overcome such drawbacks, this paper proposes an innovative multi-layered architecture to deploy heterogeneous automation and monitoring systems for microgrids. The architecture is structured into six functional layers to organize the hardware and software equipment in an integrated manner. The open protocol Modbus TCP is chosen to harmonize communications, enabling the interconnection of equipment from industrial and energy scopes, indeed of open-source nature. An experimental photovoltaic-based smart microgrid is reported as the application case to demonstrate the suitability and validity of the proposal.

https://www.mdpi.com/2071-1050/13/4/2234/pdf

Innovative Multi-Layered Architecture for Heterogeneous Automation and Monitoring Systems: Application Case of a Photovoltaic Smart Microgrid

Random modulations are treated as one of the solutions for reducing conducted electromagnetic interference (EMI) in power electronics inverters. Although in papers related to inverters, this view has not been clearly demonstrated. Especially in the state-of-the-art papers related to randomly modulated inverters, it is hard to find information about EMI measurement procedures or deeper EMC analyses. Therefore, this paper will discuss the influence of random modulations used in power electronic inverters on selected issues related to inverter EMC and power quality. A complete case study is carried out on a three-phase inverter. The undertaken analysis refers to the generation of interference, EMI measurement methodology, the conditioning of EMI filtration, and the quality of output voltage. The results show which EMI parameters are improved and which parameters are aggravated, when random modulation is used. The presented results show that random modulation should be applied in inverters with great caution. In addition, we show the limitations of EMI measurement methods and propose a more reliable way to assess the EMI spectrum.

Random Modulation in Inverters With Respect to Electromagnetic Compatibility and Power Quality

Reliable operation of the smart grid requires assurance of the electromagnetic compatibility of sensitive smart metering systems and power electronic converters, which introduce high-level electromagnetic interference. As it has been experimentally shown, currently available, normalized, and frequency domain tests are ineffective for the evaluation of data transmission error hazards. The mathematical time-domain model proposed in this paper enables assessment of the probability of errors occurring in a given transmission signal for known interference parameters. The validity of the model and its practical applicability have been experimentally verified.

Time-Domain-Based Assessment of Data Transmission Error Probability in Smart Grids With Electromagnetic Interference

Field-Programmable Gate Array (FPGA) provides the possibility to design new “electromagnetic compatibility (EMC) friendly” control techniques for power electronic converters. Such control techniques use pseudo-random modulators (RanM) to control the converter switches. However, some issues connected with the FPGA-based design of RanM, such as matching the range of fixed-point numbers, might be challenging. The modern programming tools, such as LabVIEW, may facilitate the design process, but there are still fixed-point operations and limitations in arithmetic operations. This paper presents the design insights on the FPGA-based EMC friendly control system for DC/DC converter. Probability density functions (PDF) are used to analyse and improve pseudo-random algorithms. The theoretical algorithms, hardware details and experimental results are presented and discussed in terms of conducted electromagnetic interference emission.

/pdf/fpga-based-system-for-electromagnetic-interference-2ms1sqzc23.pdf

Piotr Lezynski

Papers

Random Modulation in Inverters With Respect to Electromagnetic Compatibility and Power Quality

Time-Domain-Based Assessment of Data Transmission Error Probability in Smart Grids With Electromagnetic Interference

FPGA-Based System for Electromagnetic Interference Evaluation in Random Modulated DC/DC Converters

A novel method for EMI evaluation in random modulated power electronic converters

Pearson's random walk approach to evaluating interference generated by a group of converters