scispace - formally typeset
Search or ask a question
Proceedings ArticleDOI

Simulation of the operation of hydro plants in an electricity market using agent-based models

TL;DR: In this article, an agent-based model for an electricity market considering a detailed modeling for hydro stations is presented, and some preliminary results taking the Iberian Electricity Market as an example.
Abstract: The optimization and simulation of power systems continues to be an area of concern for electricity companies and researchers worldwide namely considering the development of electricity markets and competition in the generation activity Therefore generation companies are devoting an increasing attention to market issues justifying the development of models to help them preparing bidding strategies to the day-ahead market. In this context, agent-based models have been reported as a complement to optimization and equilibrium models when the problem is too complex to be analyzed by traditional approaches. This paper details an Agent-Based Model for an electricity market considering a detailed modeling for hydro stations and presents some preliminary results taking the Iberian Electricity Market as an example.

Summary (3 min read)

Introduction

  • Finally, their reduced response times combined with storage capability turn hydro stations as an important technology to help the management of power systems having a large share of renewable genration associated to volatile primary resources as wind and solar.
  • Agent-based simulation follows the metaphor of autonomous agents and multi-agent systems as the basis to conceptualize complex systems.
  • After this introduction, Section II overviews the Iberian Electricity Market, given that the simulation of this market is the one of the goals of this research.
  • Finally Section VI draws the most relevant conclusions.

A. New Structures and the Unbundling Model

  • To allow an appropriate development of electricity markets, significant changes were needed in power systems.
  • On the other hand, in order to nsure that the whole system operates properly, independent entities, (both at a technical and at a regulatory level) are required.
  • In general, generation and retailing are provided un er competition while transmission and distribution grid activities are organized in regulated monopolies.
  • In order to balance the demand and the supply new mechanisms have emerged, namely the day-ahead pool markets.
  • The market clearing prices are typically obtained under a marginal basis nd are usually volatile, especially in countries where hydro and other renewable energies are present in large scale (as in MIBEL).

B. The Iberian Electricity Market, MIBEL

  • Portugal and Spain power systems went through several changes in last decades.
  • This organization started to change in 1995 when a new electricity law was passed admitting the coexistence of a public and a market driven sector.
  • Later on, by the end of 1997 a new law was approved so that the Spanish electricity day-ahead market was in place in the 1st of January 1998.
  • In terms of the renewable share, both countries were very successful in increasing the amount of renewables.

A. Hydro Scheduling Optimization

  • Generation companies having hydro power plants in their portfolio have to identify the most adequate operation strategy in order to maximize their profit.
  • In a competitive environment, they have to build selling bids (and buying when they have pumping) and send them to the day-ahead market operator.
  • In addition to the uncertainty associated to the hydro conditions, the optimization of hydro power plants is a complex and nonlinear problem namely due to the nonli ear relation between the power, the flow and net head.
  • [5] uses dynamic programming but this technique usually leads to the well-known “curse of dimensionality”.
  • The mentioned nonlinear relation can also be addressed u ing an iterative procedure as described in [3].

B. Electricity Markets Modeling

  • Optimization models typically address the maximization of the revenues of a single company, often considered as a price taker.
  • Some examples were described in section III-A. Equilibrium Models represent the market behavior considering the competition between all participants.
  • More recently, Agent-Based Models became an interesting alternative when the complex level prevents using tradi ional equilibrium framework.
  • Agent-based computational economics (ACE) corresponds to the computational study of economic dynamic systems modelled as virtual worlds of interacting autonomous agents in an environment.

C. Agent-Based Models in Electricity Markets

  • There are several models in the literature addressing this issue as AMES (Agent-based Modeling of Electricity Systems), EMCAS (Electricity Market Complex Adaptive Systems) and MASCEM (Multi Agent based Electricity Market).
  • AMES is an open source platform that allows the simulation of strategic trading behaviors in restructured markets considering AC grids [10].
  • EMCAS is linked to VALORAGUA model [11] that provides longer term operation planning strategies for hydro plants.
  • Nevertheless, hydro generation, specially pumping hydro stations, is not adequately characterized taking into account the increase of renewable volatile sources.
  • Taking this into account, the main objective of this research is to simulate hydro generation in a market environment using an ABM platform, especially regarding hydro with pumping given the extra flexibility these stations have in terms of buying electricity in off peak hours when eventually extra wind generation is available and selling it in peak hours.

A. Hydro Agents

  • Hydro station agents bid their energy in the market and their strategy is very dependent on the type of reservoir and inflows.
  • This parameter is given by a learning procedure modeled using a sigmoid function that reflects the risk profile of each agent.
  • This strategy is an adaptation of the derivative-following strategy discussed in [13] and also used in [12].
  • When an agent decides to pump, it has an expectation for the next day market prices.

C. Market and System Operator Agents

  • The Market Operator agent is an artifact agent, given that it has not an associated decision making process [1].
  • It performs the market clearing operations determining the market price and communicating the market results to all market agents.
  • Regarding the System Operator, in this phase this agent is not used.

E. Regulater Agent

  • This agent monitors the generator bids and can penaliz these agents if the bid prices are very different of the marginal cost regarding thermal stations or of the water value for hydro stations.
  • The test case was based on a simplified version of the Portuguese generation system, to allow a better analysis of the results.
  • The authors considered 22 hydro power plants having constant inflows and 11 thermal (coal and natural gas) units.
  • Then the authors used the 2013 historical generation profile for these units.
  • The demand is assumed totally inelastic and prepared to pay the maximum price admitted in MIBEL, 180 €/MWh.

A. Generation companies without strategies

  • The first simulation assumes that generation companies bid without any special strategy, that is, their bids simply reflect the operation marginal cost of each station.
  • O the other hand constant profiles were used for the demand and for the wind and PV units.
  • Figure 3 shows the market price results if the generators bid their variable cost.
  • The off peak prices are determined by run of river hydro and coal plants, while the peak prices result from bids from natural gas and reservoir plans.
  • In this simulation the authors did not consider the possibility of pumping neither the monitoring action of the regulator.

B. Generation companies with strategies

  • In the second simulation the authors considered that all generators have an higher risk profile which means that they will do aggressive bids up and down (for example all increasing or decreasing the bids by 1 €/MWh in each iteration).
  • Given that the demand is completely inelastic, they rapidly realize that if they all bid the maximum price they will maximize their profits.
  • If the bid prices is limited (blue line), the market prices are more stable.
  • The results show that competition is now working and prices are very similar to the ones obtained considering the regulator limitation.
  • For this reason, in their work the authors will consider that generators have different risk profiles and the regulator will not limit the bids, but instead it will check if the bids are excessively higher regarding the marginal cost.

C. Using the 2013 renewable and demand profiles

  • The use of the real wind, PV and demand profiles for 2013 turns this simulation closer to reality.
  • Initially the authors ran a simulation using these profiles, and then the number of natural gas plants was increased to foster competition.
  • Figure 6 presents the results obtained for these two simulations.
  • As expected the markets prices are higher when wind energy is more reduced.
  • Table 1 has the annual average market prices for these simulations.

D. Comparison results without pumping hydros

  • Finally, the authors analyzed a period having a large amount of wind energy with and without pumping stations.
  • The results in Figure 7 show that if pumping is included, then zero price periods are more reduced suggesting that pumping hydro plants are able to do price arbitrage and to behave as price makers.
  • This means they have to consider the difference between forecasted and real prices as a way to minimize their risk.
  • Finally, the model will be completed including ancillary services markets namely for rese ves.

Did you find this useful? Give us your feedback

Content maybe subject to copyright    Report

Simulation of the operation of hydro plants in an
electricity market using Agent-Based Models
José Carlos Sousa
FEUP/DEEC and EDP Produção
Rua Dr. Roberto Frias,
4200-465 Porto, Portugal
jose.sousa@edp.pt
João Tomé Saraiva
FEUP/DEEC and INESC TEC
Rua Dr. Roberto Frias,
4200-465 Porto, Portugal
jsaraiva@fe.up.pt
Abstract—The optimization and simulation of power systems
continues to be an area of concern for electricity companies and
researchers worldwide namely considering the development of
electricity markets and competition in the generation activity
Therefore generation companies are devoting an increasing
attention to market issues justifying the development of models
to help them preparing bidding strategies to the day-ahead
market. In this context, agent-based models have been reported
as a complement to optimization and equilibrium models when
the problem is too complex to be analyzed by traditional
approaches. This paper details an Agent-Based Model for an
electricity market considering a detailed modeling for hydro
stations and presents some preliminary results taking the
Iberian Electricity Market as an example.
Index Terms--hydro stations, electricity markets, operation
planning, agent-based models.
I. I
NTRODUCTION
In the scope of the development of electricity markets, the
optimization of the operation of the hydro power plants has
been regaining interest both among the research community
and the electricity industry. This is certainly due to the change
of paradigm determining the operation of power systems given
the increased competition between market agents and the
increase renewable energy share in power systems. In fact,
hydro power plants have always been known for their large
reliability and availability and also given their reduced
response times. Currently, the existence of storage capabilities
in an increasing number of hydro plants turns the management
of these assets very important to the generation companies as a
well to increase the overall revenues. On the other hand, the
mentioned reduced response times turn hydro stations also
very appealing as a very efficient way to provide reserve
services so that they are becoming more and more important
from the point of view of the TSO’s. Finally, their reduced
response times combined with storage capability turn hydro
stations as an important technology to help the management of
power systems having a large share of renewable generation
associated to volatile primary resources as wind and solar.
Taking into account these concerns, it becomes important
to develop new models so that generation companies can
adequately plan the operation of hydro stations under
competition. The role of modeling and simulation models to
support decision-making in complex systems, as for example
electricity markets, has been widely established as a valid
technique. Recently, agent-based models were reported as a
complement to equilibrium models when the problem is too
complex to be analyzed by traditional models. Agent-based
simulation follows the metaphor of autonomous agents and
multi-agent systems as the basis to conceptualize complex
systems. That is, a model is built taking advantage of the
interaction between agents acting in a simulation environment.
There are several approaches in the literature to the
simulation of generation systems in market environment.
However, the presence of a large share of hydro generation,
specially pumping hydro, is not adequately treated [1].
Accordingly, this paper models the problem on an agent-based
environment and presents results for the operation of an
electricity market considering hydro stations and taking the
Iberian Electricity Market, MIBEL as the illustrative example.
We considered four types of hydro plants: run of river stations,
storage stations, pumping storage stations and pure pumping
stations. Hydro plants are modeled as agents that can produce
and also consume (in the pumping case) meaning that they
have to negotiate energy in the market as introduced in [2]. To
support the hydro-pumping decisions different optimization
models were already developed [3, 4] namely using nonlinear
programming and Genetic Algorithms.
Taking these ideas into account, this paper is structured as
follows. After this introduction, Section II overviews the
Iberian Electricity Market, given that the simulation of this
market is the one of the goals of this research. Then, Section
III gives an overview on existing approaches to deal with the
hydro scheduling with particular emphasis on agent-based
models. Section IV describes the proposed agent-based model
and Section V details the results obtained so far. Finally
Section VI draws the most relevant conclusions.
II. E
LECTRICITY MARKETS REVIEW
A. New Structures and the Unbundling Model
To allow an appropriate development of electricity
markets, significant changes were needed in power systems. In

this context, electricity shall be regarded as a product traded in
a competitive environment within certain rules. In this new
framework, companies are seen as service providers and the
grids correspond to the physical locations where electricity
markets are established. On the other hand, in order to ensure
that the whole system operates properly, independent entities,
(both at a technical and at a regulatory level) are required.
The electricity sector restructuring originated the
unbundling of the traditional vertically integrated companies
and the creation of a disaggregated structure involving
activities covering the entire value chain, namely generation,
transmission, distribution and retailing. It also includes
multiple actors as regulatory agencies, market and system
operators, and several agents in the generation and retailing
activities that are provided under competition.
In general, generation and retailing are provided under
competition while transmission and distribution grid activities
are organized in regulated monopolies. In order to balance the
demand and the supply new mechanisms have emerged,
namely the day-ahead pool markets. The day-ahead markets
that exist in several European countries correspond to short
term forward markets based on the matching of the selling and
buying bids for each hour of the next day. The market clearing
prices are typically obtained under a marginal basis and are
usually volatile, especially in countries where hydro and other
renewable energies are present in large scale (as in MIBEL).
In order to be aware of this volatility, longer term contracts are
also possible under different horizons and conditions.
B. The Iberian Electricity Market, MIBEL
Portugal and Spain power systems went through several
changes in last decades. In Portugal, the power industry was
nationalized in the 1970s with the creation of a vertically
integrated utility. This organization started to change in 1995
when a new electricity law was passed admitting the
coexistence of a public and a market driven sector. Later, in
2006 a new electricity law was passed organizing the industry
in generation, transmission, distribution and retailing. The
Regulatory Agency was created in 1995 and is responsible for
the publication of several codes and for setting the regulated
tariffs. Since 2007, all clients are eligible and the free market
represented 73% of the demand by the end of 2013.
In Spain the power system was also organized in terms of
vertically integrated utilities having a regional distribution.
Then a new law was also passed in 1995 in a first attempt to
introduce some competitive mechanism in the system. Later
on, by the end of 1997 a new law was approved so that the
Spanish electricity day-ahead market was in place in the 1st of
January 1998. Since then, a fast transition of regulated captive
clients to the free market was implemented so that full
eligibility was achieved in 2003.
The implementation of MIBEL started with the signature
of a memorandum by the Portuguese and Spanish
governments in 2001. After several delays, a common bilateral
contract trading mechanism was set in place in 2006 and the
joint day-ahead market started in the 1st of July 2007 as an
extension of the already existing Spanish day-ahead market. In
the first operation years the electricity prices in the two areas
were different in a large number of hours due to the
application of market splitting to solve congestion in the
interconnections. Nowadays, due to the increase of the
interconnection capacity and the increasing share of
generation in distribution networks, transmission grids are less
loaded so that the number of congested hours declined. As a
result the prices in the two countries converged to common
values in almost 85% of the hours in 2013 and 2014.
Regarding the generation mix, both countries have a large
share of hydro plants with a huge variation in their annual
output. In terms of the renewable share, both countries were
very successful in increasing the amount of renewables. This
corresponded to a strategic policy adopted by successive
governments to use more intensively endogenous resources, to
enlarge the energetic independency and also to develop new
industrial activities thus creating new jobs. By the end of
2014, wind power reached an installed capacity of 5270 MW
out of 17827 MW in Portugal (30%) and of 22854 MW out of
102259 MW in Spain (22 %) with a contribution to supply the
demand of 25% in Portugal and 21% in Spain.
III. L
ITERATURE REVIEW ON
H
YDRO
S
CHEDULING
A. Hydro Scheduling Optimization
Generation companies having hydro power plants in their
portfolio have to identify the most adequate operation strategy
in order to maximize their profit. In a competitive
environment, they have to build selling bids (and buying when
they have pumping) and send them to the day-ahead market
operator. In addition to the uncertainty associated to the hydro
conditions, the optimization of hydro power plants is a
complex and nonlinear problem namely due to the nonlinear
relation between the power, the flow and net head. The
literature includes a large number of publications on this topic.
In this scope, [5] uses dynamic programming but this
technique usually leads to the well-known “curse of
dimensionality”. Other publications use mixed integer linear
programming [6] or meta-heuristics, as Simulated Annealing
[7], Neural Networks [8] or Genetic Algorithms [4]. The
mentioned nonlinear relation can also be addressed using an
iterative procedure as described in [3].
B. Electricity Markets Modeling
There are several works that were developed to model
electricity markets using different techniques that can be
organized in four main areas [9]:
Optimization problems, addressing a single company
also known as single firm optimization models;
Equilibrium Models based on Game Theory,
considering a larger number of competitors;
Agent-Based Models, ABM, that simulate the behavior
of the companies and the interactions between agents;
Hybrid solutions.
Optimization models typically address the maximization
of the revenues of a single company, often considered as a
price taker. Some examples were described in section III-A.
Equilibrium Models represent the market behavior
considering the competition between all participants. More
recently, Agent-Based Models became an interesting

alternative when the complex level prevents using traditional
equilibrium framework. Agent-based computational
economics (ACE) corresponds to the computational study of
economic dynamic systems modelled as virtual worlds of
interacting autonomous agents in an environment.
C. Agent-Based Models in Electricity Markets
There are several models in the literature addressing this
issue as AMES (Agent-based Modeling of Electricity
Systems), EMCAS (Electricity Market Complex Adaptive
Systems) and MASCEM (Multi Agent based Electricity
Market). AMES is an open source platform that allows the
simulation of strategic trading behaviors in restructured
markets considering AC grids [10]. EMCAS is a commercial
ABM software developed by the Argone National Lab having
the capability of taking decentralized decision-making along
with learning and adaptation for agents. An EMCAS
simulation includes both the end users and the demand
companies from whom they purchase electricity. EMCAS is
linked to VALORAGUA model [11] that provides longer term
operation planning strategies for hydro plants. With this
information, EMCAS uses the price forecasts and weekly
hydro schedules given by VALORAGUA to provide intra-
week hydro plant optimization for hourly supply offers.
Finally, the MASCEM is a simulation platform based on a
multi-agent framework [12]. It includes agents with strategies
for bid definition, acting in forward, day-ahead, and balancing
markets and considering both simple and complex bids turning
it both in a short and a medium term model.
Nevertheless, hydro generation, specially pumping hydro
stations, is not adequately characterized taking into account
the increase of renewable volatile sources. For instance,
EMCAS includes the VALORAGUA model turning it very
dependent on the performance of the VALORAGUA. This
also means that EMCAS does not include the definition of
bidding strategies to hydro power plants. Taking this into
account, the main objective of this research is to simulate
hydro generation in a market environment using an ABM
platform, especially regarding hydro with pumping given the
extra flexibility these stations have in terms of buying
electricity in off peak hours when eventually extra wind
generation is available and selling it in peak hours. This will
allow us to study their impact on systems having a large
penetration of renewable sources, especially wind.
IV. D
EVELOPED
A
GENT
-B
ASED
M
ODEL
As mentioned before the main goal of this paper is to
present a model for hydro plants agents in an Agent-Based
framework, based on the model introduced in [1] and in [2].
A. Hydro Agents
Hydro station agents bid their energy in the market and
their strategy is very dependent on the type of reservoir and
inflows. Depending on the hydro type, the bidding price
strategy is determined by the water value on the reservoir, by
a learning parameter α and by a decision supporting tool, all
of them originally described in [2] and modeled as (1). In a
first approach, this bid price has the same value for every
hour of the next day, except for pure pumping plants. The
water value function f(water value) provides each plant with a
reference bid price that changes every day depending on the
reservoir level, as illustrated in Figure 1. This means that if
the level is larger, then the value of the water stored is more
reduced and so a more reduced biding price can also be used.
This water value function is calculated for each weak
according to the procedure detailed in [2].
Bid price strategy = f(water value)+bid up/down (α) (1)
Figure 1. Base bidding taking into account the water value.
The bid up/down (α) parameter models the strategy of
each agent by increasing or decreasing its bid price as a way
to increase the profit. This parameter is given by a learning
procedure modeled using a sigmoid function that reflects the
risk profile of each agent. If an agent has a higher risk profile,
the bid range will be larger. On the other hand, a low risk
profile will lead to a lower bid range as illustrated in Figure 2
for hydro agents having different risk profiles. This strategy
is an adaptation of the derivative-following strategy discussed
in [13] and also used in [12]. A derivative follower does
incremental increases (or decreases) in price, continuing to
move its price in the same direction until the observed
profitability level falls. At this point, the direction of the
movement is reversed. In future works, this strategy will be
combined will a Q-learning procedure.
Figure 2. Bidding strategy taking into account the risk profile of each agent.
The developed ABM model 4 considers four types of
hydro agents having different bidding strategies:
Run of river – these agents typically have a water value
function near 0, so they will have more focus on their
bid up/down strategy;
Storage these agents will have a bid value directed
related to their water value function as well as to their
bid up/down strategy;
Storage with pumping the bid price is linked to their
water value function and a bid up/down strategy. They
also have the possibility of buying energy to pump
water to their reservoir, taking advantage of low prices;
Pure pumping these agents are assigned a zero water
value because these reservoirs are usually small. They
will use decision support tools to forecast the day-
ahead electricity prices so that they can define an
arbitrage strategy based on price differential between
peak and off peak hours. Given that forecasted and real
market prices can differ, the energy used in pumping is
limited by a parameter β. This parameter is updated
Max bid up
Max bid down
Strategy
Max bid up
Max bid down
Strategy
bid
Reservoir
Max
Min

along the simulation and it reflects the relation between
the forecasted prices and the real market prices. When
an agent decides to pump, it has an expectation for the
next day market prices. But in real time there is a risk
that on generation periods the real price is below the
expected one and on pumping periods the real price is
above the expected one turning pumping less
profitable. Therefore, this parameter is updated to
address this risk using a learning procedure.
B. Thermal and Renewable Generation Agents
These agents have a strategy similar to the hydro agents,
but the water value function is substituted by their marginal
cost. Renewable agents bid at 0 €/MWh in order to model
their dispatch priority according to the Portuguese legislation.
C. Market and System Operator Agents
The Market Operator agent is an artifact agent, given that
it has not an associated decision making process [1]. It
performs the market clearing operations determining the
market price and communicating the market results to all
market agents. Regarding the System Operator, in this phase
this agent is not used. In future developments, it will manage
the ancillary service markets, namely to determine the amount
of secondary and tertiary reserve to procure and contract.
D. Inelastica and Elastic Consumer Agents
These include two types of agents: inelastic agents that
buy energy at the maximum value allowed in the MIBEL rules
(180 €/MWh), and elastic agents that are designed to model
the behaviour of consumers that can directly participate in the
market, typically large industries or hydro pumping stations.
Elastic consumers will be responsible for some demand
response regarding price variations in their buying curves.
E. Regulater Agent
This agent monitors the generator bids and can penalize
these agents if the bid prices are very different of the marginal
cost regarding thermal stations or of the water value for hydro
stations. It also has the possibility of imposing a limit to the
bidding prices so that they adhere more closely to the marginal
cost of thermal stations or to the water value of hydro stations.
V. P
RELIMINARY
R
ESULTS
The test case was based on a simplified version of the
Portuguese generation system, to allow a better analysis of the
results. We considered 22 hydro power plants having constant
inflows and 11 thermal (coal and natural gas) units. The
generation mix also includes 5 reservoir pumping plants. In a
first step, wind and PV units were set constant for all hours at
2000 MW. Then we used the 2013 historical generation
profile for these units. The demand is assumed totally inelastic
and prepared to pay the maximum price admitted in MIBEL,
180 €/MWh. Initially, the demand has a daily constant profile
and then we used the 2013 demand data.
A. Generation companies without strategies
The first simulation assumes that generation companies
bid without any special strategy, that is, their bids simply
reflect the operation marginal cost of each station. On the
other hand constant profiles were used for the demand and for
the wind and PV units. Figure 3 shows the market price results
if the generators bid their variable cost. The off peak prices are
determined by run of river hydro and coal plants, while the
peak prices result from bids from natural gas and reservoir
plans. In this simulation we did not consider the possibility of
pumping neither the monitoring action of the regulator.
Figure 3. Market price results for simulation 1.
B. Generation companies with strategies
In the second simulation we considered that all generators
have an higher risk profile which means that they will do
aggressive bids up and down (for example all increasing or
decreasing the bids by 1 €/MWh in each iteration). Figure 4
shows the market price results. Although there is no
communication between generators, after same time all of
them start biding close to 180 €/MWh, which is the maximum
price the demand is prepared to pay. Given that the demand is
completely inelastic, they rapidly realize that if they all bid the
maximum price they will maximize their profits.
Figure 4. Market prices results for simulation 1.
However, in real markets this doesn’t happen because not
all generators behave in the same way and there is a regulator
to monitor and eventually penalize them. Figure 5 presents
the results admitting that the Regulator Agent limits the bids
by 20% of the generators marginal cost, as well as the results
considering that generators have different risk profiles so that
they can change they bid price along the simulation. If the bid
prices is limited (blue line), the market prices are more stable.
However, we are introducing an artificial limit in the
simulation that has an impact in competition. The red line
represents the results considering that the regulator doesn’t
limit the bid prices but generators have different bid
strategies, that is different risk profiles. The results show that
competition is now working and prices are very similar to the
ones obtained considering the regulator limitation. In this
case, a consecutive bid up made by a risky generator trying to
maximize the profit may not be successful because the other
generators have different risk profiles and do not follow the
same strategy. For this reason, in our work we will consider
that generators have different risk profiles and the regulator
will not limit the bids, but instead it will check if the bids are
excessively higher regarding the marginal cost. If that
happens, the profit of the associated agents is penalized.

Figure 5. Market prices results for different risk profiles.
C. Using the 2013 renewable and demand profiles
The use of the real wind, PV and demand profiles for 2013
turns this simulation closer to reality. Initially we ran a
simulation using these profiles, and then the number of natural
gas plants was increased to foster competition. Figure 6
presents the results obtained for these two simulations.
Figure 6. Market prices results for 2013 wind and demand profile.
As expected the markets prices are higher when wind
energy is more reduced. In fact on the right side of Figure 6,
wind energy is more reduced and market prices are close to 90
€/MWh when the higher marginal operation cost is 75
€/MWh. In blue we can also observe the impact of having
more competition on peak periods due to more natural gas
power plants and leading to peak price reductions. Table 1 has
the annual average market prices for these simulations.
Table 1. Average market prices using the 2013 profiles.
Simulation with 2013 wind, PV
and demand profiles
Average annual Market
Price (€/MWh)
Bid at the marginal operation cost 41.22
All Generators with higher risk 171.08
Generators with different risk profiles 47.02
More natural gas units 45.13
D. Comparison results without pumping hydros
Finally, we analyzed a period having a large amount of
wind energy with and without pumping stations. The results
in Figure 7 show that if pumping is included, then zero price
periods are more reduced suggesting that pumping hydro
plants are able to do price arbitrage and to behave as price
makers. This means they have to consider the difference
between forecasted and real prices as a way to minimize their
risk. In this case, this risk was modeled as detailed in [5].
Figure 7. Impact in market prices of having pumping hydro.
VI. C
ONCLUSIONS
This paper presents the preliminary results of an ABM
model to simulate the electricity market focusing on modeling
hydro units. The obtained results confirm the agents have
learning capabilities and are maximizing their profit. We also
observe the importance of regulation in this kind of markets
and that competition is able to decrease the market prices
namely when considering more CCGT’s. On the other hand,
having an accurate model for hydro pumping units is very
important in the Portuguese case because these units often
behave as price makers. In future works, this model will be
extended to include the Spanish generation system and a Q-
learning process to improve the learning capabilities of the
generation agents. Finally, the model will be completed
including ancillary services markets namely for reserves.
A
CKNOWLEDGMENT
The authors thank for the contributions of Z. Kokkinogenis, R.
Rossetti and A. P. Rocha from LIAC - Artificial Intelligence
and Computers Sciences Laboratory (FEUP).
R
EFERENCES
[1] J. C. Sousa, Z. Kokkinogenis, R. Rossetti, J. T. Saraiva, “Electricity
Market Modeling and Renewable Energy Integration: an Agent-Based
Conceptual Model”, in Proc.10th Int. Multidisciplinary Modeling &
Simulation Multiconference, I3M/SESDE 2013, Athens, Sept. 2013.
[2] J. C. Sousa, J. T. Saraiva, Z. Kokkinogenis, R. Rossetti, “Operation
Planning of Hydro Power Plants Using Agent Based Models”, in Proc.
of MedPower 2014, Athens, Nov. 2014.
[3] J. C. Sousa, V. T. Mendes, J. T. Saraiva, “Estimation of the
Remuneration of Hydro Plants in a Market Environment Using an
Iterative Under-Relaxation Approach”, in Proc. 2009 IEEE Bucharest
Power Tech, Romenia, 28 June – 2 July 2009.
[4] Guerreiro, C.A., J. T. Saraiva, J. C. Sousa, V. T. Mendes, “Operation
planning of hydro stations using genetic algorithms considering their
impact on the electricity market prices”, in Proc. 11
th
International
European Electricity Market Conference, EEM´14, Kracow, May 2014.
[5] S.-C. Chang, C.-H. Chen, I.-K. Fong, P. B. Luh, “Hydroelectric
Generation Scheduling With An Effective Differential Dynamic
Programming Algorithm”, IEEE Transactions on Power Systems, vol. 5,
no. 3, pp. 737-743, August 1990.
[6] G. W. Wang, C. T. Su, “A Practical Mixed Integer Linear Programming
Based Short Term Hydro Scheduling”, in Proc. of the IEEE Asia-Pacific
Trans. and Dist. Conference, vol. 3, pp. 1606-1610, October 2002.
[7] K. Wong, Y. Wong, A Parallel Simulated Annealing Algorithm for
Short-Term Hydro Scheduling”, in Proc. of the IEEE Int Forum on
Applications of Neural Networks Power Systems, pp. 335-340, April
1993.
[8] R. H. Liang, Y. Y. Hsu, “Short-Term Hydro-Scheduling using Hopfield
Neural Network”, IEE Proc. Generation, Transmission & Distribution,
vol. 143, no. 3, pp. 269-275, May 1996.
[9] G. Li, J. Shi, X. Qu, "Modeling methods for GenCo bidding strategy
optimization in the liberalized electricity spot market–A state-of-the-art
review", Energy, vol. 36, no. 8, pp. 4686-4700, August 2011.
[10] H. Li, L. Tesfatsion, “Development of open source software for power
market research: The AMES test bed”, Journal of Energy Markets, vol.
2, no. 2, pp. 111-128, 2009.
[11] P. Thimmapuram, T. D. Veselka, V. Koritarov, S. Vilela, R. Pereira, R.
F. Silva, “Modeling hydro power plants in deregulated electricity
markets: Integration and application of EMCAS and VALORAGUA”, in
Proc. EEM2008, pp. 1-6, May 2008.
[12] Z. Vale, T. Pinto, I. Praça, H. Morais, “MASCEM: electricity markets
simulation with strategic agents”, IEEE Transactions on Intelligent
Systems, vol. 26, no. 2, pp. 9-17, March-April 2011.
[13]A. Greenwald, J. Kephart, and G. Tesauro, “Strategic Pricebots
Dynamics,” in Proceedings of the 1
st
ACM Conf. Electronic Commerce,
ACM Press, pp. 58–67, 1999.
Citations
More filters
Proceedings ArticleDOI
21 May 2019
TL;DR: A review of the literature on different applications of ABM in electricity markets is provided in this paper, where it is shown that ABMs have been applied to a wide range of studies of different parts of electricity trading.
Abstract: The complexity of modeling electricity markets is increasing, due to for example increased penetration of renewable energy sources, electric vehicles and more active participation from the demand side. An analysis of a system with multiple market participants displaying certain behavior, could be difficult through an optimization problem. Due to several advantages, agent based approach can be followed to model the behavior of different market participants in the electricity markets. Lately, considerable amount of research work has been carried out on developing Agent-Based Models (ABMs) for electricity markets. This paper is aimed at providing a review of the literature on different applications of ABM in electricity markets. It is shown that ABMs have been applied to a wide range of studies of different parts of electricity trading. Some specific electricity markets modeled with ABMs have also been mentioned. According to the literature survey, the research gap is highlighted and future scope of work in this area is discussed.

9 citations


Cites background from "Simulation of the operation of hydr..."

  • ...which was discussed in [23] and [24] in the context of MIBEL....

    [...]

  • ...In [23], they found that accurate model for hydropower is important in Portuguese market as they often behave as price maker....

    [...]

  • ...[23] Design an electricity market considering hydropower stations Hydro, thermal, renewable gen, market, system operators, consumers, regulator Bidding strategy is function of water in reservoir, learning parameter of agent,decision support tool DA...

    [...]

Proceedings ArticleDOI
06 Jun 2016
TL;DR: In this paper, an agent-based simulation of the electricity market is extended to investigate load flows in transmission grid systems, and an AC load flow approach is incorporated to investigate the resulting load flow in the transmission grid in Germany.
Abstract: Europe's transmission grids are confronted with increasing numbers of congestions due to multiple reasons leading to rising distances of generation and demand, such as renewable energy sources, which substitute generation from thermal and hydraulic power. Furthermore, planned measures of grid expansion are frequently delayed, resulting in higher flows within the existing network topology which need to be handled by grid operation measures. Due to these changes, the consideration of the impact of market results on the transmission grid is of rising importance when investigating load flows in transmission grid systems. In this paper, an existing agent-based simulation of the electricity market is extended. First, the integration of electricity storage with hydro power plants is improved by a hydro dispatch optimisation, accounting for cascading of multiple units with multiple reservoirs. Second, an AC load flow approach is incorporated to investigate the resulting load flows in the transmission grid in Germany.

7 citations


Cites background from "Simulation of the operation of hydr..."

  • ...Traditionally, deterministic hydro dispatch has been a subject of linear mixed-integer optimisation models such as [6] and has been integrated recently into an agent-based approach [7], which does...

    [...]

Proceedings ArticleDOI
06 Jun 2016
TL;DR: In this article, an agent-based approach to model the day-ahead electricity market having a particular emphasis on hydro generation is described. And the introduction of the Q-learning algorithm in the model as a way to enhance the performance of generation agents.
Abstract: The restructuring of power systems with the introduction of electricity markets and decentralized structures increased the number of participating entities. This is particularly true in generation and retailing which are now provided under competition. Accordingly, it is important to develop models to simulate the behavior of these agents and to optimize their participation in electricity markets. Among them, it is essential to adequately model generation agents namely in countries having a large share of hydro stations. This paper describes an agent-based approach to model the day-ahead electricity market having a particular emphasis on hydro generation. Apart from the characterization of the agents, the paper details the introduction of the Q-Learning algorithm in the model as a way to enhance the performance of generation agents. This paper also presents some preliminary results taking the Portuguese generation system as an example.

4 citations


Cites background or methods from "Simulation of the operation of hydr..."

  • ...As mentioned before the main goal of this paper is to introduce a Q learning procedure in the Agent-Based Hydro plants model detailed in [1], [2] and [3]....

    [...]

  • ...The developed ABM model considers four types of hydro agents having different bidding strategies [3] as briefly outlined below:...

    [...]

  • ...Depending on the type of hydro unit, the bidding price strategy is determined by the water value on the reservoir, by a learning parameter α and by a decision supporting tool, all of them originally described in [2, 3] and modeled by (2)....

    [...]

  • ...Hydro plants are modeled as agents that can produce and also consume (in the pumping case) meaning that they have to negotiate energy in the market as introduced in [2-3]....

    [...]

  • ...Accordingly, this paper presents a model using an agent-based environment that was originally described in [2-3] and in which we are now introducing a Q learning procedure....

    [...]

Journal ArticleDOI
30 Apr 2021
TL;DR: This paper provides a review of the literature regarding ABM in power systems followed by an analysis in more detail regarding specific applications that are becoming relevant in this new paradigm.
Abstract: In the last two decades, power systems have experienced several changes, mainly related to organizational and operational restructuring. The appearance of new actors contributes to developing new business models and modifies its traditional operation activities. As a direct result, there is a need for new control solutions and strategies to integrate these different players. Agent-Based Models (ABM) have been increasingly used to model complex systems since they are especially suited to model systems influenced by social interactions between flexible, autonomous, and proactive agents. This paper provides a review of the literature regarding ABM in power systems followed by an analysis in more detail regarding specific applications that are becoming relevant in this new paradigm.

4 citations


Cites methods from "Simulation of the operation of hydr..."

  • ...In Sousa and Saraiva (2017), an ABM model is described using Q-learning to provide knowledge for the agents to select their decision....

    [...]

Proceedings ArticleDOI
01 Jun 2017
TL;DR: In this article, an enhanced model for the Short Term Hydro Scheduling Problem, HSP, that includes the impact of operation decisions on the market prices and the possibility of adjusting the tailwater level and the generation and pumping efficiencies as a function of the flow is presented.
Abstract: This paper describes an enhanced model for the Short Term Hydro Scheduling Problem, HSP, that includes the impact of operation decisions on the market prices and the possibility of adjusting the tailwater level and the generation and pumping efficiencies as a function of the flow. The solution approach uses an iterative procedure that solves in each iteration a linearized HSP problem using the linprog function of the MATLAB® Optimization Toolbox and that updates the value of the head to be used in the next iteration. The paper also includes results from a realistic Case Study based on the cascade of 9 hydro stations (4 of them with pumping) installed in the Portuguese section of the Douro River.

1 citations


Cites background from "Simulation of the operation of hydr..."

  • ...The introduction of competition in activities as generation and retailing justifies the development of new models and tools to ensure that the most adequate decisions are adopted [1, 2]....

    [...]

References
More filters
Journal ArticleDOI
01 Aug 2011-Energy
TL;DR: In this paper, the authors present a comprehensive literature analysis on the state-of-the-art research of bidding strategy modeling methods, including game theory, mathematical programming, game theory and agent-based models.

195 citations


"Simulation of the operation of hydr..." refers background in this paper

  • ...There are several works that were developed to model electricity markets using different techniques that can be organized in four main areas [9]:...

    [...]

Journal ArticleDOI
TL;DR: An effective multiplier method-based differential dynamic programming (DDP) algorithm for solving the hydroelectric generation scheduling problem (HSP) is presented and results demonstrate the efficiency and optimality of the algorithm.
Abstract: An effective multiplier method-based differential dynamic programming (DDP) algorithm for solving the hydroelectric generation scheduling problem (HSP) is presented. The algorithm is developed for solving a class of constrained dynamic optimization problems. It relaxes all constraints but the system dynamics by the multiplier method and adopts the DDP solution technique to solve the resultant unconstrained dynamic optimization problem. The authors formulate the HSP of the Taiwan power system and apply the algorithm to it. Results demonstrate the efficiency and optimality of the algorithm for this application. Computational results indicate that the growth of the algorithm's run time with respect to the problem size is moderate. CPU times of the testing cases are well within the Taiwan Power Company's desirable performance; less than 30 minutes on a VAX/780 mini-computer for a one-week scheduling. >

152 citations

Journal ArticleDOI
TL;DR: A new methodology integrated in MASCEM for bid definition in electricity markets is proposed, which uses reinforcement learning algorithms to let players perceive changes in the environment, thus helping them react to the dynamic environment and adapt their bids accordingly.
Abstract: To study and understand this type of market, we developed the Multiagent Simulator of Competitive Electricity Markets (MASCEM) platform based on multiagent simulation. The MASCEM multiagent model includes players with strategies for bid definition, acting in forward, day-ahead, and balancing markets and considering both simple and complex bids. Our goal with MASCEM was to simulate as many market models and player types as possible. This approach makes MASCEM both a short and medium term simulation as well as a tool to support long-term decisions, such as those taken by regulators. This article proposes a new methodology integrated in MASCEM for bid definition in electricity markets. This methodology uses reinforcement learning algorithms to let players perceive changes in the environment, thus helping them react to the dynamic environment and adapt their bids accordingly.

152 citations


"Simulation of the operation of hydr..." refers methods in this paper

  • ...Th is strategy is an adaptation of the derivative-following strategy discussed in [13] and also used in [12]....

    [...]

  • ...Finally, the MASCEM is a simulation platform based on a multi-agent framework [12]....

    [...]

Posted Content
TL;DR: In this article, the authors discuss potential benefits and drawbacks of developing Open Source Software (OSS) for power market research, using the AMES Wholesale Power Market Test Bed for concrete illustration.
Abstract: Open Source Software (OSS) expresses the idea that developers should be able to license the publication of their software in a manner permitting anyone to freely use, modify, and distribute the software. Today OSS is widely used in the software industry, such as for language development tools (e.g., NetBeans for Java), office document processors (e.g., OpenOffice), and operating systems (e.g., Linux, OpenSolaris).Yet OSS has been slow to penetrate the power industry; heavy reliance is still placed on closed-source commercial software packages. The OSS in use tends to be for specialized purposes (e.g., circuit design) rather than for the general-purpose analysis of power systems. This study discusses potential benefits and drawbacks of developing OSS for power market research, using the AMES Wholesale Power Market Test Bed for concrete illustration. AMES downloads, tutorials, and research publications can be accesssed at http://www2.econ.iastate.edu/tesfatsi/AMESMarketHome.htm

101 citations

Proceedings ArticleDOI
01 Nov 1999
TL;DR: A comparative study of four candidate price-setting strategies that meet informational and computational requirements: gametheoretic pricing (GT), myoptimalpricing (MY), derivative following (DF), and Q-learning (Q), which exhibits superior performance to all the others.
Abstract: Shopbots are software agents that automatically query multiple sellers on the Internet to gather information about prices and other attributes of consumer goods and services. Rapidly increasing in number and sophistication, shopbots are helping more and more buyers minimize expenditure and maximize satisfaction. In response at least partly to this trend, it is anticipated that sellers will come to rely on pricebots, automated agents that employ price-setting algorithms in an attempt to maximize pro ts. This paper reaches toward an understanding of strategic pricebot dynamics. More speci cally, this paper is a comparative study of four candidate price-setting strategies that di er in informational and computational requirements: gametheoretic pricing (GT), myoptimalpricing (MY), derivative following (DF), and Q-learning (Q). In an e ort to gain insights into the tradeo s between practicality and pro tability of pricebot algorithms, the dynamic behavior that arises among homogeneous and heterogeneous collections of pricebots and shopbot-assisted buyers is analyzed and simulated. In homogeneous settings | when all pricebots use the same pricing algorithm | DFs outperform MYs and GTs. Investigation of heterogeneous collections of pricebots, however, reveals an incentive for individual DFs to deviate to MY or GT. The Q strategy exhibits superior performance to all the others since it learns to predict and account for the long-term consequences of its actions. Although the current implementation of Q is impractically expensive, techniques for achieving similar performance at greatly reduced computational cost are under investigation.

88 citations


"Simulation of the operation of hydr..." refers methods in this paper

  • ...Th is strategy is an adaptation of the derivative-following strategy discussed in [13] and also used in [12]....

    [...]

Frequently Asked Questions (16)
Q1. What have the authors contributed in "Simulation of the operation of hydro plants in an electricity market using agent-based models" ?

In this context, agent-based models have been reported as a complement to optimization and equilibrium models when the problem is too complex to be analyzed by traditional approaches. This paper details an Agent-Based Model for an electricity market considering a detailed modeling for hydro stations and presents some preliminary results taking the Iberian Electricity Market as an example. 

The electricity sector restructuring originated the unbundling of the traditional vertically integrated companies and the creation of a disaggregated structure involving activities covering the entire value chain, namely generation, transmission, distribution and retailing. 

By the end of 2014, wind power reached an installed capacity of 5270 MW out of 17827 MW in Portugal (30%) and of 22854 MW out of 102259 MW in Spain (22 %) with a contribution to supply the demand of 25% in Portugal and 21% in Spain. 

The Regulatory Agency was created in 1995 and is responsible for the publication of several codes and for setting the regulated tariffs. 

These include two types of agents: inelastic agents that buy energy at the maximum value allowed in the MIBEL rules (180 €/MWh), and elastic agents that are designed to model the behaviour of consumers that can directly participate in the market, typically large industries or hydro pumping stations. 

In a competitive environment, they have to build selling bids (and buying when they have pumping) and send them to the day-ahead market operator. 

It also has the possibility of imposing a limit to the bidding prices so that they adhere more closely to the marginal cost of thermal stations or to the water value of hydro stations. 

The day-ahead markets that exist in several European countries correspond to short term forward markets based on the matching of the selling and buying bids for each hour of the next day. 

In the first operation years the electricity prices in the two areaswere different in a large number of hours due to the application of market splitting to solve congestion in the interconnections. 

The first simulation assumes that generation companies bid without any special strategy, that is, their bids simply reflect the operation marginal cost of each station. 

Although there is no communication between generators, after same time all of them start biding close to 180 €/MWh, which is the maximum price the demand is prepared to pay. 

Some examples were described in section III-A. Equilibrium Models represent the market behavior considering the competition between all participants. 

On the other hand, in order to ensure that the whole system operates properly, independent entities, (both at a technical and at a regulatory level) are required. 

In blue the authors can also observe the impact of having more competition on peak periods due to more natural gas power plants and leading to peak price reductions. 

It also includes multiple actors as regulatory agencies, market and system operators, and several agents in the generation and retailing activities that are provided under competition. 

Depending on the hydro type, the bidding price strategy is determined by the water value on the reservoir, by a learning parameter α and by a decision supporting tool, all of them originally described in [2] and modeled as (1).