Modeling of Suppliers' Learning Behaviors in a Market Environment

TLDR: In this article, an anticipatory reinforcement learning technique is used to model the learning behaviors of electricity suppliers in a Day-Ahead electricity market, where the market is modeled as a multi-agent system with interacting agents including supplier agents, Load Serving Entities and a Market Operator.

Abstract: An important objective of electricity suppliers is to maximize their profits over a planning horizon and comply with the market rules. This objective requires suppliers to learn from their bidding experience and behave in an anticipatory way. With volatile Locational Marginal Prices (LMPs), ever-changing transmission grid conditions, and incomplete information about other market participants, decision making for a supplier is a complex task. A learning algorithm that does not require an analytical model of the complicated market but allows suppliers to learn from experience and act in an anticipatory way is a suitable approach to this problem. Q-Learning, an anticipatory reinforcement learning technique, has all these desired properties. Therefore, it is used in this research to model the learning behaviors of electricity suppliers in a Day-Ahead electricity market. The Day-Ahead electricity market is modeled as a multi-agent system with interacting agents including supplier agents, Load Serving Entities and a Market Operator. Simulation of the market clearing results under the scenario in which agents have learning capabilities is compared with the scenario where agents report true marginal costs. It is shown that, with Q- Learning, electricity suppliers are making more profits compared to the scenario without learning due to strategic gaming. As a result, the LMP at each bus is substantially higher.