While reinforcement learning (RL) is gaining popularity in energy systems control, its real-world applications are limited due to the fact that the actions from learned policies may not satisfy functional requirements or be feasible for the underlying physical system. In this work, we propose PROjected Feasibility (PROF), a method to enforce convex operational constraints within neural policies. Specifically, we incorporate a differentiable projection layer within a neural network-based policy to enforce that all learned actions are feasible. We then update the policy end-to-end by propagating gradients through this differentiable projection layer, making the policy cognizant of the operational constraints. We demonstrate our method on two applications: energy-efficient building operation and inverter control. In the building operation setting, we show that PROF maintains thermal comfort requirements while improving energy efficiency by 4% over state-of-the-art methods. In the inverter control setting, PROF perfectly satisfies voltage constraints on the IEEE 37-bus feeder system, as it learns to curtail as little renewable energy as possible within its safety set.

Enforcing Policy Feasibility Constraints through Differentiable Projection for Energy Optimization

For restructured electricity industries undergoing energy transition, designing effective and efficient frequency control arrangements is a complex and ongoing task that requires appropriate configuration of controllers, generator technical connection requirements, market arrangements and wider policy settings. In this paper, we provide an overview and assessment of these arrangements in Australia’s National Electricity Market - a useful case study given its long-standing frequency control ancillary services markets, yet recent challenges in maintaining secure frequency control. We assess the performance of these evolving arrangements in delivering improved frequency control outcomes, with particular regard to growing renewable penetrations and evident tensions between mandatory requirements and market-based incentives. Based on this assessment, we draw out four key insights on designing frequency control arrangements as power system capabilities and needs change: (1) Understanding control action interactions, (2) Implementing efficient price formation and cost-allocation mechanisms, (3) Monitoring and assessing service provision to better align participant remuneration with service quality, and (4) Considering both regulatory and market mechanisms and their consequences and interactions. In particular, we discuss the trade-offs between effective and efficient outcomes, and provide arguments for more robust and forward-looking frequency control arrangements during energy transition. 

Insights on designing effective and efficient frequency control arrangements from the Australian National Electricity Market

Local voltage control in distribution networks using PI control of active and reactive power

This paper proposes a novel reactive power-based control strategy for single-phase PV inverters (PVIs) to simultaneously improve voltage unbalance (VU) and voltage regulation (VR) in low-voltage distribution networks. The proposed strategy relies on communication links between neighboring PVIs to exchange limited data. In this strategy, each PVI finds communication paths between itself and the closest neighboring ones connected to other phases. Then, using the obtained paths and the maximum and the minimum voltage magnitude of the grid, PVIs improve both VU and VR at the same time. The performance of the proposed control strategy is evaluated by various simulation studies using the IEEE European low-voltage test feeder and considering different operational conditions. In addition, the impacts of moving clouds and a failure in the communication links have been assessed. The simulation results exhibit that using the proposed control strategy, the voltage magnitude of all the nodes will remain within the allowed limits and at the same time, the phase voltage unbalance factor will be also significantly improved. 

Improving Voltage Regulation and Unbalance in Distribution Networks Using Peer-to-Peer Data Sharing Between Single-Phase PV Inverters

Designing a Bi-Level Flexibility Market for Transmission System Congestion Management Considering Distribution System Performance Improvement

It's a sunny spring DAY in South Australia. A light breeze is cooling the coastal state capital of Adelaide as approximately 260,000 distributed solar photovoltaic (D-PV) systems on residential and business rooftops generate electricity, setting a new state record for the lowest minimum electrical demand for the third time this season. All looks peaceful throughout the suburbs. D-PVs form an irregular yet persistent patchwork across one-third of the houses in the state. Most consumers are going about their day without a glance at the D-PV panels on these roofs. However, the mild temperature and bountiful sunshine lead the D-PV generation to climb, causing the voltage to creep higher and power to flow in reverse across large swathes of the distribution network. Some D-PVs are tripping off, unbeknownst to their owners and the local network operator.

Peter Kilby

Papers

Consumer-Led Transition: Australia's World-Leading Distributed Energy Resource Integration Efforts