Agent-based homeostatic control for green energy in the smart grid

TL;DR: A novel control mechanism that models and controls a system comprising of a green energy supplier operating within the grid and a number of individual homes, and a new carbon-based pricing mechanism that takes advantage of carbon-intensity signals available on the Internet in order to provide real-time pricing.

Abstract: With dwindling nonrenewable energy reserves and the adverse effects of climate change, the development of the smart electricity grid is seen as key to solving global energy security issues and to reducing carbon emissions. In this respect, there is a growing need to integrate renewable (or green) energy sources in the grid. However, the intermittency of these energy sources requires that demand must also be made more responsive to changes in supply, and a number of smart grid technologies are being developed, such as high-capacity batteries and smart meters for the home, to enable consumers to be more responsive to conditions on the grid in real time. Traditional solutions based on these technologies, however, tend to ignore the fact that individual consumers will behave in such a way that best satisfies their own preferences to use or store energy (as opposed to that of the supplier or the grid operator). Hence, in practice, it is unclear how these solutions will cope with large numbers of consumers using their devices in this way. Against this background, in this article, we develop novel control mechanisms based on the use of autonomous agents to better incorporate consumer preferences in managing demand. These agents, residing on consumers' smart meters, can both communicate with the grid and optimize their owner's energy consumption to satisfy their preferences. More specifically, we provide a novel control mechanism that models and controls a system comprising of a green energy supplier operating within the grid and a number of individual homes (each possibly owning a storage device). This control mechanism is based on the concept of homeostasis whereby control signals are sent to individual components of a system, based on their continuous feedback, in order to change their state so that the system may reach a stable equilibrium. Thus, we define a new carbon-based pricing mechanism for this green energy supplier that takes advantage of carbon-intensity signals available on the Internet in order to provide real-time pricing. The pricing scheme is designed in such a way that it can be readily implemented using existing communication technologies and is easily understandable by consumers. Building upon this, we develop new control signals that the supplier can use to incentivize agents to shift demand (using their storage device) to times when green energy is available. Moreover, we show how these signals can be adapted according to changes in supply and to various degrees of penetration of storage in the system. We empirically evaluate our system and show that, when all homes are equipped with storage devices, the supplier can significantly reduce its reliance on other carbon-emitting power sources to cater for its own shortfalls. By so doing, the supplier reduces the carbon emission of the system by up to 25p while the consumer reduces its costs by up to 14.5p. Finally, we demonstrate that our homeostatic control mechanism is not sensitive to small prediction errors and the supplier is incentivized to accurately predict its green production to minimize costs.

Summary

1. INTRODUCTION

• This is an important challenge because the integration of renewable energy generators requires a significant shift from the traditional principle of “supply follows demand” whereby generators always have to keep up with demand in the grid.
• Such a pricing scheme also means that, should demand be higher than can be met by the green supplier, the greenest form of energy from the grid (where a mix of green and non-green suppliers also exist) will be bought by the supplier to meet excess demand.
• As the authors show in Section 6, their mechanism is robust to errors in prediction and hence, as better prediction technology is developed, the benefits accrued from using their mechanism will only improve.

2. BACKGROUND AND RELATED WORK

• The issue of integrating renewable energy supplies into the grid to reduce carbon emissions while ensuring their intermittency does not have an adverse impact (e.g., by requiring more spinning reserves or causing brownouts) has been actively researched for a number of years.
• Who foresaw that improvements in communication technologies would pave the way for new control mechanisms to coordinate energy suppliers with users in order to account for shortfalls (e.g., by asking them to reduce consumption) or excesses (e.g., by letting them consume more) in energy supplies in real-time [Schweppe et al. 1980].
• ACM Transactions on Intelligent Systems and Technology, Vol. V, No. N, Article A, Publication date: January 2011.
• Moreover, in their recent work, the authors showed how such agents can be modelled to individually optimise the storage of electricity given a real-time price and how this led to demand on the grid being flattened (without explicitly exchanging plans among agents) [Vytelingum et al. 2010; Ramchurn et al. 2011].
• Thus, if consumers are charged for their carbon emissions, they will optimise their behaviour to reduce their emissions, and hence reduce the tax paid.

3. GREEN SUPPLIER AND AGENT MODELS

• Specifically, Figure 1 depicts the main elements of the system involving the supplier and the consumers.
• Thus, the grid (consisting of electricity markets and physical networks) and weather (e.g., wind speed or sunshine) act as external influences on them.
• Smaller slots (e.g., 5 or 15 minutes) could be considered but half-hour slots are used here as this is a common interval used within electricity meters and electricity markets.

3.1. The Green Supplier

• The authors model the green supplier (e.g., with wind turbines and/or solar power generators) as having a pool of consumers that subscribe to it for electricity.
• 15 The supplier can predict, with a reasonable 12The UK government has planned to equip all 26M houses in the country with a smart meter by 2020 while in the US it is expected that about 60M houses will be equipped by the same date.
• Hence, the green supplier needs to ensure that it minimises the amount of extra capacity it needs to buy in spot markets by getting its consumers to follow its production pattern.

3.2. The Consumer Agents

• Here, the authors describe their model of the consumer agent, which builds upon and extends a recent model for homes equipped with smart meters [Vytelingum et al. 2010].
• This electricity can either be bought from the supplier or retrieved from a storage device.
• This would mean adding a hard constraint on the consumers’ demand and/or price very expensively any consumption above the level provided by the grid.
• ACM Transactions on Intelligent Systems and Technology, Vol. V, No. N, Article A, Publication date: January 2011.
• The storage efficiency αa models the fact that if qi kWh amount of energy is stored, then only αaqi kWh may be subsequently used.

4. THE CARBON-BASED PRICING MECHANISM

• To communicate electricity prices, the authors develop a novel carbon-based pricing scheme that the supplier sends to its customers.
• The scheme relies on the availability of a real-time carbon intensity signal (measured in gCO2/kWh and representing the amount of CO2 emitted for every unit of energy consumed) that is broadcast on the internet16 by national grid operators (as in the UK and Ireland).
• Now, as with other examples of pricing mechanisms (e.g., fixed pricing, time-of-use, or real-time pricing), the retail prices must be calculated to reflect the supplier’s retail margin profit and its exposure to the risk of peak prices in the wholesale market.
• Through the above pricing scheme the supplier incentivises consumers to use the green energy (since it always cheaper) it produces rather than grid energy.

5. HOMEOSTATIC CONTROL

• Given the variability in weather conditions affecting the generation capacity, the system, comprising of the agents and the supplier, needs to continuously adjust to these conditions to maximise its efficiency.
• This needs to happen given that both the supplier and the consumers aim to maximise their individual profits.
• More importantly, they can only do so if they communicate to ensure that the aggregate demand from all the consumers is as close as possible to the real-time supply from the producer, while ensuring that any extra energy needed is bought at times when the carbon intensity of the grid is the lowest.
• While the sensing and prediction of short-term weather conditions can be easily implemented using existing machine learning techniques [Alpaydin 2004], significant challenges lie in determining the appropriate control signal to be sent to the agents to ensure that their behaviour in reaction to such a signal meets the above requirements.
• The authors address these challenges and detail the different elements of their solution , in the following subsections.

5.1. The Signal to the Agents

• Using this information, an agent can then manage its demand in order to both minimise its costs and also its carbon emissions.
• While this ACM Transactions on Intelligent Systems and Technology, Vol. V, No. N, Article A, Publication date: January 2011.
• The signal allows the consumer to compute its cost of electricity based on their pricing mechanism outlined earlier on.
• In Section 5.3 the authors will show how to adapt this signal to consider the feedback that the supplier gets from the population of agents.
• If an agent overuses (above the target), it increases the expected cost of electricity as the supplier would need to acquire the additional demand at pgridi prices.

5.2. The Agents’ Behaviour

• Constraints 2 and 3 decouple the amount consumed by the agent in terms of the green and non-green components since the green component (i.e., l a,greeni ) is limited by the supplier according to its signal γi through γil a,t i (in Constraint 3).
• Moreover, load profiles tend to change across seasons and for special days (e.g., Christmas or for special sporting events).
• ACM Transactions on Intelligent Systems and Technology, Vol. V, No. N, Article A, Publication date: January 2011.
• Now, not all agents may possess storage capacity (or some may not have enough storage to cover their whole consumption) and this is not known to the supplier a priori.

5.3. The Adaptive Mechanism

• The aim of the adaptive mechanism is to learn the optimal signal to send to the agents (given their response to a previously sent signal).
• The aim here would be to tell the agents to adjust their loads (using the battery bai ) so that γi ideally converges to 1 (i.e., demand following supply).
• As can be seen, using the adaptive signal, the homeostatic control mechanism tries to minimise the difference between green energy produced and current total demand based on a learning rate βS which determines the rate at which it will change γi to reflect the feedback it receives.
• The supplier signals its consumers to reduce their use of green energy at time i (and ensuring that γ i is never negative).

6. EVALUATION

• The aim of this evaluation is to empirically demonstrate the effectiveness of the homeostatic control mechanism presented in the previous section in inducing demand to follow supply such that the carbon emissions of the system is reduced.
• The authors adopt an empirical approach here, as opposed to an analytical approach seeking equilibrium characterisations, in order to evaluate the system under a wider variety of settings than would be feasible within a theoretical framework, 24 and predict equilibrium outcomes (for differently parameterised versions of their mechanism) without overly restrictive assumptions.
• As the authors will see, despite the utility-maximising behaviour of consumers with storage capability, the system quickly converges to the optimal behaviour.
• In what follows, the authors first detail their experimental setup and then go on to evaluate the efficiency of their homeostatic control mechanism (against the optimal behaviour) in terms of green energy use and costs of electricity, in particular for different proportions of the population with storage capability.
• The authors focus on the proportions with storage capacity to identify the critical amount of storage needed in the population to achieve the maximum returns on investment in batteries for individual users and for the population as a whole.

6.1. Experimental Setup

• The authors simulate a pool of 5000 consumers26 subscribed to a green supplier that is able to supply 50% of its demand from its own renewable sources and the remainder bought from the grid.
• Due to UK data protection issues, such results cannot be reproduced here.
• Finally, the demand of each consumer is modelled on the real UK data profile of the average user on the Domestic Unconstrained Tariff (i.e., the typical UK average profile before any demand side management technologies, such as storage, are used [Vytelingum et al. 2010]) and the simulation is run 500 times, each run lasting 100 days.
• The consumers with storage capabilities are thus able to optimise their storage profile to reduce their costs (and implicitly reduce their carbon emissions as a result of their carbon-based pricing scheme).

6.2. Efficiency of Homeostatic Control

• First, the authors analyse the daily effect of homeostatic control on the system.
• From Figure 6, given a population with 50% having storage capability, the authors can see a daily convergence of their homeostatic control system with an adaptive mechanism, for different learning rates βS , to the optimal solution at 91.8%.
• As can be seen, the non-adaptive version is not very efficient (i.e., only 88.3%) as it overestimates the proportion of the population with storage and understates γ that represents the amount of green energy available to consumers with storage.
• When the supplier adopts the adaptive mechanism using a higher learning rate (i.e., β S = 0.05), this leads to a faster, though less smooth, convergence as opposed to a slower, but smoother convergence to the lower learning rate (i.e., βS = 0.005).
• Also, an efficient outcome means that the consumers have lower carbon emissions overall since they maximise their use of renewable energy (and thus, minimise their use of grid energy).

6.3. Effect of the Population Size with Storage Capability

• Based on the previous experiment, here the authors set the learning rate at βS = 0.005 (to ensure convergence) for their homeostatic control mechanism and vary the proportion of the population with storage capability from 0 to 1.
• Now, when given a signal about green production (as in the homeostatic control mechanisms), the consumers are significantly greener as they are optimising their demand based on how much and when green energy is available to them.
• When storage penetration is at its highest, the authors can see how the effect of homeostatic control peaks with up to a 25% increase in green energy usage efficiency (i.e., a 25% decrease in carbon emissions) compared to without homeostatic control.
• In addition, Figure 7(b) (log-scale) also compares the system, at equilibrium, with and without homeostatic control against the optimal solution (i.e., the authors are comparing the relative error between the optimal solution and where the system converges to).
• ACM Transactions on Intelligent Systems and Technology, Vol. V, No. N, Article A, Publication date: January 2011.

6.4. The Cost of Electricity

• Here the authors use their carbon-based pricing scheme to evaluate the cost of electricity given that different proportions of the population are able to utilise the full capacity of the green supplier.
• Furthermore, the cost without homeostatic control is marginally higher than with homeostatic control.
• Moreover, because the payoff of owning storage (rather than not) is always higher, it is likely that all consumers will eventually acquire storage devices and capitalise on the supplier’s control signal of green energy.
• ACM Transactions on Intelligent Systems and Technology, Vol. V, No. N, Article A, Publication date: January 2011.
• Specifically, the authors assume a storage device typically costing £200 per kWh and a start-up cost of £200 (i.e., installation cost, wiring and voltage inverter), with a lifetime of 10 years.

6.5. Effect of Green Energy Prediction Error

• In so doing, the total demand will be higher than the green energy available at peak times (when consumers are most incentivised to use green energy because of the greater cost of grid electricity) such that the supplier will need to buy from the grid to cover the difference, leading to higher costs.
• When the supplier slightly overstates γ, the authors can clearly see that cost increases exponentially ) while efficiency only increases by up to a negligible 0.5% ).
• Thus, crucially to minimise cost (and trade-off only up to 0.5%), the supplier is incentivised to predict accurately since poor prediction (whether understating or overstating) invariably results in rapidly increasing costs for its consumers.
• When the absolute prediction error is small, the increase in costs is also small such that the system is relatively non-sensitive to small prediction errors.

7. CONCLUSIONS

• In this paper the authors have presented a novel decentralised homeostatic control mechanism for a green supplier and its consumers connected through the grid.
• Given this, the authors have shown how agents can optimise their consumption of energy by storing electricity at times when it is cheap, thereby maximising their individual utility and minimising their carbon footprint.
• Thus, the authors are able to show that their mechanism can achieve up to 25% greater efficiency for the supplier and up to ACM Transactions on Intelligent Systems and Technology, Vol. V, No. N, Article A, Publication date: January 2011.
• While this may be fine for some people, others may not trust their agents to act on their behalf in such a sensitive domain.
• This, the authors believe, will require more advanced control techniques given that carbon intensity signals may disappear altogether and a key objective of suppliers will change from maximising green energy usage to ensuring that demand always follows supply throughout the grid in order to avoid brown outs and black outs.

Figures

Fig. 2. Relationships between carbon intensity and the price of electricity against demand on the balancing market (Source: NETA, UK August-September 2009).

Fig. 5. The adaptive mechanism given a chosen learning rate βS and inputs qti and l A,t i .

Fig. 9. Effect of inaccurate green generation prediction. With no homeostatic control, cost is 10% more than the optimal cost throughout, as is the efficiency of green usage at 79.5%.

Fig. 4. Mathematical program used by an agent to find its optimal storage strategy.

Fig. 6. Efficiency of green energy usage for the system with and without the adaptive mechanism. The system with no homeostatic control is only 79.5% efficient throughout.

Fig. 1. Here we illustrate the electricity and information flows between the actors in the system we consider. Specifically, our model specifies the price plan used by the supplier, the heterogeneous storage devices used by the consumers as well as the decision making involved in charging or using their battery. We also assume that the grid provides the carbon intensity signals which we use in our mechanism but do not consider how the electricity is transmitted and delivered to consumers by the grid.

Fig. 7. Green energy usage efficiency with and without homeostatic control for different proportions of the population with storage. The relative deviation from the optimal solution (as computed in Equation (2)) is given in (b).

Fig. 3. Flow of information in the system involving the supplier and the consumers connected to the grid. Building on the system presented in Figure 1, consumers receive two signals; a carbon intensity signal ρi from the grid (which they use to predict the next day’s carbon intensity) and (in addition to the price plan shown in Figure 1) γt+1i (for the next day) computed by the supplier using qi (i.e., its production for the current day) and lAi (i.e., the overall consumption for the current day). The consumers then optimise their electricity usage given the signal γt+1i and using their battery. The supplier is informed of the amount consumed by all its customers (i.e., lAi ), hence closing the loop. Note that we do not show the payments the supplier receives based on splitting the consumption of its customers in terms of their green and non-green components (i.e., lAi = l A,green

Fig. 8. Average cost of electricity for storage-capable consumers with and without homeostatic control for different proportions of the population with storage.

Full text

A
Agent-Based Homeostatic Control
for Green Energy in the Smart Grid
SARVAPALI D. RAMCHURN, PERUKRISHNEN VYTELINGUM,
ALEX ROGERS, and NICHOLAS R. JENNINGS
Intelligence, Agents, Multimedia Group
School of Electronics and Computer Science
University of Southampton
Southampton, SO17 1BJ, UK
{sdr,pv,acr,nrj}@ecs.soton.ac.uk
With dwindling non-renewable energy reserves and the adverse effects of climate change, the development
of the smart electricity grid is seen as key to solving global energy security issues and to reducing carbon
emissions. In this respect, there is a growing need to integrate renewable (or green) energy sources in the
grid. However, the intermittency of these energy sources requires that demand must also be made more
responsive to changes in supply, and a number of smart grid technologies are being developed, such as high-
capacity batteries and smart meters for the home, to enable consumers to be more responsive to conditions
on the grid in real-time. Traditional solutions based on these technologies, however, tend to ignore the
fact that individual consumers will behave in such a way that best satisfies their own preferences to use
or store energy (as opposed to that of the supplier or the grid operator). Hence, in practice, it is unclear
how these solutions will cope with large numbers of consumers using their devices in this way. Against this
background, in this paper, we develop novel control mechanisms based on the use of autonomous agents to
better incorporate consumer preferences in managing demand. These agents, residing on consumers’ smart
meters, can both communicate with the grid and optimise their owner’s energy consumption to satisfy their
preferences. More specifically, we provide a novel control mechanism that models and controls a system
comprising of a green energy supplier operating within the grid and a number of individual homes (each
possibly owning a storage device). This control mechanism is based on the concept of homeostasis whereby
control signals are sent to individual components of a system, based on their continuous feedback, in order
to change their state so that the system may reach a stable equilibrium. Thus, we define a new carbon-based
pricing mechanism for this green energy supplier that takes advantage of carbon-intensity signals available
on the internet in order to provide real-time pricing. The pricing scheme is designed in such a way that
it can be readily implemented using existing communication technologies and is easily understandable by
consumers. Building upon this, we develop new control signals that the supplier can use to incentivise agents
to shift demand (using their storage device) to times when green energy is available. Moreover, we show
how these signals can be adapted according to changes in supply and to various degrees of penetration of
storage in the system. We empirically evaluate our system and show that, when all homes are equipped with
storage devices, the supplier can significantly reduce its reliance on other carbon-emitting power sources to
cater for its own shortfalls. By so doing, the supplier reduces the carbon emission of the system by up to
25% while the consumer reduces its costs by up to 14.5%. Finally, we demonstrate that our homeostatic
control mechanism is not sensitive to small prediction errors and the supplier is incentivised to accurately
predict its green production to minimise costs.
Categories and Subject Descriptors: I.2.11 [Computing Methodologies]: Artificial Intelligence—Distributed Artificial In-
telligence
General Terms: Agents, Multi-Agent Systems
Additional Key Words and Phrases: Computational Sustainability, Electricity, Multi-Agent Systems, Agent-Based Control,
Agents.
1. INTRODUCTION
Achieving energy security and reducing carbon emissions have been recognised as two of the most
important challenges of this century given the rapid depletion of worldwide oil and gas reserves
and the potentially devastating effects of climate change [DECC 2009a; US Department Of Energy
2003]. To this end, the creation of a smart electricity grid has been advocated as a key component in
ACM Transactions on Intelligent Systems and Technology, Vol. V, No. N, Article A, Publication date: January 2011.

A:2 Ramchurn, Vytelingum, Rogers, and Jennings
delivering efficient low-carbon energy. In particular, the vision of a smart grid includes technologies
that reduce losses in the transmission of electricity, that integrate intermittent renewable energy
generators (such as wind turbines and solar panels), and that attempt to reduce or shift demand by
providing real-time information about consumption and prices using smart meters in homes.
In this paper we focus in particular on the pressing problem of integrating renewable energy
generation into the smart grid. This is an important challenge because the integration of renewable
energy generators requires a significant shift from the traditional principle of “supply follows de-
mand” whereby generators always have to keep up with demand in the grid. To date, this has been
acceptable only because the output from non-renewable energy sources (such as coal or gas fired
power stations) can be turned up or down at will. In contrast, renewable energy generators (i.e.,
green suppliers) are sensitive to seasons and weather conditions and are therefore variable and in-
termittent in their output. This means that they cannot be powered up at will to meet the demand,
nor is it possible to accurately predict exactly how much energy they will generate. While, from a
technical point of view, integrating renewable energy requires the development of technologies to
ensure power flows in a controlled manner from renewable energy generators to the grid, from an
economic point of view, renewable energy suppliers need to devise new strategies to trade effec-
tively in the electricity markets in which they operate in order to ensure that they can make a profit
while still satisfying the demand from their consumers given the intermittency in their supply.
When the supplier is not able to satisfy the demand of its customers from its own renewable gen-
eration capacity, it must typically buy additional electricity at short notice (usually incurring a high
cost which reduces its potential profits) from the wholesale electricity market within which it oper-
ates [Kirschen and Strbac 2004]. Conversely, when the supplier produces more than its customers
demand, it must sell this extra electricity at a lower price than what it would typically obtain from
its customers.
1
It is therefore crucial for the supplier to predict its production accurately in order to
bid effectively in markets, but more importantly, the supplier should be able to influence the demand
from its customers to mitigate the impact of low production.
Now, while the issue of predicting renewable energy production and bidding in electricity mar-
kets has been actively researched in the power systems domain [Kirschen and Strbac 2004; Milligan
et al. 2009; Morales et al. 2010], there is relatively little work studying how to influence the demand
of residential consumers (see Section 2 for more details). Typical approaches to demand manage-
ment involve the supplier sending a predicted real-time price, via the consumers’ smart meter, to
incentivise them to increase or decrease demand [Faruqui and George 2005]. These mechanisms
assume that by feeding more information to consumers, the latter will be able to understand the
conditions on the grid (e.g., generator outages or a high price for electricity at peak time) and react
to such conditions by switching devices on or off, or changing their thermostat settings on their
heaters (or air conditioners), and in future, use electricity stored in their home or electric car battery.
However, such an assumption is challenged by the fact that the signals from the grid, aside from be-
ing rather complex for the average user [Schweppe et al. 1989], require the user to perform complex
calculations in order to optimise her consumption and storage of electricity (i.e., to know what is the
best time to store or use electricity). Given this, in this paper, we advocate the use of autonomous
software agents as a decentralised control paradigm whereby each owner would have her own agent,
running on her smart meter, advising on the best consumption pattern she could adopt to achieve
her goals (e.g., minimising carbon emissions and/or costs). A key advantage of this approach is that
consumers do not need to be experts at energy trading or scheduling in order to maximise their
benefits, therefore their adoption allows for a larger impact in implementing demand management
strategies based on complex signals from the grid.
1
In general, a supplier acts as the middle-man between the generators from which it buys electricity and the customers to
which it sells electricity. However, we consider the case of the generator and the supplier being the same actor as is often the
case in the UK or US electricity market, particularly with regards to ‘green’ focused suppliers such as Ecotricity (UK), New
England GreenStart, or Green Mountain Energy (US).
ACM Transactions on Intelligent Systems and Technology, Vol. V, No. N, Article A, Publication date: January 2011.

Agent-Based Homeostatic Control for Green Energy in the Smart Grid A:3
In more detail, agents can be endowed with the ability to learn their owner’s preferences from
their consumption history and can run optimisation algorithms that solve complex scheduling tasks
(e.g., heating cycles of boilers or charging profiles of their electric car battery) with very little direct
input, if at all, from their owner. In so doing, these agents can minimise their owner’s cost, given
price signals from the grid, by efficiently scheduling the on/off cycles of deferrable devices (such
as washing machines or boilers) or the storage profiles of batteries (i.e., storing when prices are
low and using the stored electricity when prices are high). However, even if a real-time price signal
is provided by the grid or supplier, agents have no indication as to how much they should change
their usage (by deferring loads) or store electricity in order to meet the capacity of the renewable
energy supplier. This is important because, if, for example, the agents are incentivised to consume
more, based on a low predicted price, the aggregate might end up consuming much more than can
be provided by the supplier. Conversely, given a high price, the collection of agents might consume
much less than what the supplier produces leaving the supplier with unused green energy. Hence, in
general, relying solely on the price signal may result in a mismatch between the production capacity
of the supplier and the demand from the consumers that might require the supplier to buy extra
capacity at a high price or sell its extra production at a loss on the wholesale electricity market. The
supplier may, in turn, pass on such costs to its customers and also increase the consumers’ carbon
emissions as it uses carbon-emitting generators (from which it buys on the wholesale market) to
supply their extra demands. Moreover,providingonly a price signal ignoresthe factthat some agents
may also be aiming to reduce their owner’s carbon emissions (as there is a growing population of
such green users [Kockar et al. 2009]) since such a signal ignores the carbon intensity (i.e., the
carbon emissions per unit of energy) of the energy being supplied. Given these issues with real-
time pricing in this domain, it is crucial that better control mechanisms are developed to mitigate
the occurrence of such mismatches in order to make the integration of renewable energy into the
smart grid more profitable and more reliable. In particular, given the reaction of the agents to such
signals, the controller (i.e., supplier) should be able to provide feedback to the agents in order for
them to adjust their consumption profile while still allowing them to minimise their costs and their
carbon emissions. The challenge here is to generate such signals and stabilise the system without
the supplier being able to fully observe the state of the individual agents in terms of their profile
of electricity usage. This challenge is further exacerbated by the fact that the agents may not wish
to reveal their battery storage capacity (which determines by how much they can defer demand to
follow supply) as they may not trust the provider to use this information to their benefit.
To meet this challenge, in this paper, we introduce the concept of agent-based homeostatic con-
trol to coordinate a green supplier and its consumers, who may own storage capability, within the
smart grid.
2
In particular, we model a system composed of a green energy supplier (e.g., a wind
and/or solar powered generator) providing electricity to a large number of consumers whose aims
are to reduce their individual costs and carbon emissions. To capture the consumers’ preferences
and goals, we develop agents that can optimise their owner’s battery storage profile according to
signals from the grid. Our model aims to capture real-life settings where, for example, a third-party
wind turbine/photovoltaic array provides energy to a small number of houses or buildings, or where
a regional/national supplier serves customers across a wide area. We then provide a novel control
mechanism to ensure that demand from the agents follows the production from the green supplier.
Our approach is inspired from the principle of homeostasis [Marieb and Hoehn 2007]. The latter
describes a living organism’s adjustment of its internal interdependent components to maintain a
stable constant condition and was first used in the context of the electricity grid by Schweppe et
al. [1980]. The analogy between our system and a living organism is appropriate given that our
system is based on the three underpinning concepts of homeostasis: sensing, sending a control sig-
2
Our approach can be generalised to consider loads such as washing machines or boilers that can be deferred to achieve
results similar to using storage, but this is beyond the scope of this paper.
ACM Transactions on Intelligent Systems and Technology, Vol. V, No. N, Article A, Publication date: January 2011.

A:4 Ramchurn, Vytelingum, Rogers, and Jennings
nal, and feedback. First, our system is able to sense and predict
3
the effect of external conditions
(i.e., weather in our case) using widely available weather sensors and satellite imagery. Second,
the supplier can compute a control signal to be sent to the agents based on the impact of weather
conditions on its production of electricity. Third, through the resulting response of the agents in the
system (i.e., the combined effect of its load and storage profiles), feedback is provided to the sup-
plier which decides how to adapt its control signal to ensure the system reaches the state it desires.
Through several iterations of the above process, the system aims to converge to a more efficient
solution (see Section 6). While the idea of homeostatic control in this domain was pioneered more
than 30 years ago, it was not deemed practical at the time due to the lack of communication and
distributed computing technologies that would allow large populations of consumers of different
types (of different income and usage levels) to participate in such mechanisms. However, with the
prevalence of broadband communications
4
and the development of agent-based technologies for a
wide variety of applications including the energy domain [Jennings et al. 1996; Bussmann et al.
2004], it is now possible to revisit this proposition and instantiate homeostatic control mechanisms
within the smart grid.
In more detail, this paper advances the state of the art in the following ways:
(1) We implement a novel real-time carbon-based pricing mechanism that builds on the carbon
intensity signals provided by the grid (e.g., the National Grid in the UK or Eire Grid in Ireland
already provide this data in real-time) through the internet.
5
Using this mechanism, suppliers,
using renewable energysources or not, do not need to install additional hardware to compute and
transmit a real-time price to consumers as it can easily be obtained over an existing broadband
connection. We show that by setting a price plan to match the carbon intensity of the grid,
suppliers can effectively align the carbon-based prices to the real-time prices they effectively
trade at on the grid. Such a pricing scheme also means that, should demand be higher than can
be met by the green supplier, the greenest form of energy from the grid (where a mix of green
and non-green suppliers also exist) will be bought by the supplier to meet excess demand. By
so doing, carbon-sensitive consumers are incentivised to subscribe to the supplier since they
can minimise both their costs and carbon emissions at the same time. In turn, this helps the
supplier reduce its reliance on other grid energy sources with high carbon emissions (for which
the supplier may be taxed according to the market regime it operates within [Kockar et al. 2009;
Hobbs et al. 2010]). It is also important to note that, our pricing scheme is not restricted to
just a green supplier as our prices (except the price of energy strictly obtainable from the green
supplier) are only dependent on the conditions on the grid (i.e., grid carbon intensity reflecting
generation costs). Thus, our pricing scheme could potentially be used by any supplier trying
to appeal to an environmentally conscious consumer aiming to reduce her contribution to the
overall carbon emissions of the grid.
(2) We develop a novel control signal which ensures that agents are incentivised to store a given
quantity of electricity according to changes in green energy generation capacity caused by
changing weather conditions. The signal provides an estimate to the agent of how much it should
aim to change its storage behaviour in order to minimise its costs.
(3) We develop algorithms, based on mathematical programming, for agents to optimise their en-
ergy storage, given the control signal they receive, in order to minimise their costs, and in so
doing, their carbon emissions.
3
Note that we do not develop novel sensing and prediction algorithms as this is beyond the scope of this work. Rather, we
assume such predictions are available and focus on building the control mechanism that uses such predictions. However, as
we show in Section 6, our mechanism is robust to errors in prediction and hence, as better prediction technology is developed,
the benefits accrued from using our mechanism will only improve. Future work will study such prediction algorithms in more
detail.
4
Google PowerMeter (http://www.google.com/powermeter) and AlertMe (http://www.alertme.com)
systems already utilise home broadband links in order to monitor energy consumption over the web.
5
See http://www.bmreports.com and http://www.ideasproject.info/software.
ACM Transactions on Intelligent Systems and Technology, Vol. V, No. N, Article A, Publication date: January 2011.

Agent-Based Homeostatic Control for Green Energy in the Smart Grid A:5
(4) We provide a novel adaptive mechanism for the green supplier to learn the best signal to send to
the agents in order to maximise its efficiency. In particular, the mechanism works even when the
supplier is unawareof the proportionof the consumers in the system that possess enough storage
to cope with the variability in supply (which cannot be accurately predicted by the supplier).
(5) We empirically evaluate our approach and show that our agent-based homeostatic control with
our novel adaptive mechanism (compared to a traditional real-time pricing mechanism) results
in up to 25% improvementin efficiency (which maximises the use of green energy by its pool of
consumers and hence reduces their carbon emissions) for the supplier and up to 14.5% savings
for the agents. Furthermore, we show how errors in the supplier predicting its green production
can result in higher costs or carbon emissions. In particular, we show that the supplier is incen-
tivised to accurately predict its green production to minimise its costs and that our homeostatic
control mechanism is not sensitive to small prediction errors.
The rest of this paper is structured as follows. Section 2 presents the general background of our
work and outlines the rationale behind our design choices. Section 3 describes the model of the
green supplier and the agents, each endowed with their own preferences and their battery storage
capabilities. Given this, in Section 4, we present our novel carbon-based pricing mechanism. Then,
Section 5 details our homeostatic control mechanism. Section 6 evaluates our mechanism and Sec-
tion 7 concludes.
2. BACKGROUND AND RELATED WORK
The issue of integrating renewable energy supplies into the grid to reduce carbon emissions while
ensuring their intermittency does not have an adverse impact (e.g., by requiring more spinning
reserves or causing brownouts) has been actively researched for a number of years. The seminal
idea of using homeostatic control to account for intermittency (for any energy source type) was
initially suggested by Schweppe et al. who foresaw that improvements in communication technolo-
gies would pave the way for new control mechanisms to coordinate energy suppliers with users in
order to account for shortfalls (e.g., by asking them to reduce consumption) or excesses (e.g., by
letting them consume more) in energy supplies in real-time [Schweppe et al. 1980]. Such mech-
anisms would then allow demand to follow supply. Their work, however, remained preliminary
and, as such, untested under simulated settings. In particular, they did not model the individual be-
haviour of agents (in terms of their basic preferences and how they optimised their consumption)
and did not consider the issue of carbon emissions to be important at the time. Moreover, they
used the frequency of alternating current (AC) as a vehicle for real-time pricing. In the absence of
other communication mechanisms, this choice was probably the only viable one. However, while
frequency is a good indicator of generation failing to meet demand, it does not correspond to the
cost of electricity in real-time electricity markets [Schweppe et al. 1989]. In contrast, our more
up-to-date approach is able to exploit recent communication and computational advances in order
to model individual agent behaviours and develop pricing mechanisms and control signals that be
communicated in real-time while still matching the current market conditions.
Since Schweppe’s work, most research in the power systems community has focused on facilitat-
ing the integration of intermittent sources of energy by providing more reliable predictions and con-
trol of energy generation from such sources [Milligan et al. 2009; Morales et al. 2010] or by match-
ing renewable energy generators with high-capacity electricity storage providers (either attached to
wind turbines or performing arbitrage in an electricity market) [Burgio et al. 2009; Lubosny and
Bialek 2007; Bathurst and Strbac 2003; Makarov et al. 2010; Korpaas et al. 2003]. Hence, most of
this work has focused on the use of large ‘utility-scale’ batteries controlled by one user as opposed
to small domestic batteries (e.g., electric cars or water heaters
6
) where each battery is individually
owned and where the owner is only intent on maximising her own preferences. This is mainly due
to the fact that coordinating a large number of individual users (owning batteries) with suppliers
6
Note that the storage of hot water or air is effectively equivalent to the storage of electrical energy in some settings.
ACM Transactions on Intelligent Systems and Technology, Vol. V, No. N, Article A, Publication date: January 2011.

Frequently asked questions

Q1. What contributions have the authors mentioned in the paper "A agent-based homeostatic control for green energy in the smart grid" ?

Against this background, in this paper, the authors develop novel control mechanisms based on the use of autonomous agents to better incorporate consumer preferences in managing demand. More specifically, the authors provide a novel control mechanism that models and controls a system comprising of a green energy supplier operating within the grid and a number of individual homes ( each possibly owning a storage device ). Thus, the authors define a new carbon-based pricing mechanism for this green energy supplier that takes advantage of carbon-intensity signals available on the internet in order to provide real-time pricing. Moreover, the authors show how these signals can be adapted according to changes in supply and to various degrees of penetration of storage in the system. The authors empirically evaluate their system and show that, when all homes are equipped with storage devices, the supplier can significantly reduce its reliance on other carbon-emitting power sources to cater for its own shortfalls. 5 %. Finally, the authors demonstrate that their homeostatic control mechanism is not sensitive to small prediction errors and the supplier is incentivised to accurately predict its green production to minimise costs. Building upon this, the authors develop new control signals that the supplier can use to incentivise agents to shift demand ( using their storage device ) to times when green energy is available. 

Q2. What are the future works in "A agent-based homeostatic control for green energy in the smart grid" ?

In future the authors aim to investigate the properties of their control mechanism within a more theoretical framework in order to prove its convergence properties ( under specific settings ) and expand their optimisation model to consider other types of constraints ( e. g., bounds on energy storage to extend battery life or decay in battery storage capacity ). Another important future direction involves extending the model to consider multiple green suppliers operating in the grid such that they constitute the majority of the energy producers. While in their simulations the authors assumed that the renewable power generation does not vary significantly across days, this may not always be the case. While the authors have studied how prediction errors on the supplier side affect the system, they have simplified the model of the consumers to avoid significant prediction errors. 

Q3. What is the way to reduce the costs of an agent?

Since the authors assume that an agent’s daily consumption (load) profile is fixed (i.e., price does not affect the loads in a single time slot but can affect stored or discharged energy), an agent a can only try to minimise its costs by storing energy when prices are low and using as much of that energy as possible when prices are high. 

Q4. What is the effect of limited storage on the system?

In particular, with limited storage, agents are less able to react to the variability in supply (communicated through the γ signals). 

Q5. What is the main assumption of the approach?

a major assumption of their approach is that users will be happy to delegate the task of shifting energy to their agents in the hope to get the best deal. 

Q6. What is the effect of homeostatic control on the consumption of green energy?

When storage penetration is at its highest, the authors can see how the effect of homeostatic control peaks with up to a 25% increase in green energy usage efficiency (i.e., a 25% decrease in carbon emissions) compared to without homeostatic control. 

Q7. How does the system perform when the prediction error is small?

when the absolute prediction error is small, the increase in costs is also small such that the system is relatively non-sensitive to small prediction errors. 

Q8. What is the main focus of the research in the power systems community?

Since Schweppe’s work, most research in the power systems community has focused on facilitating the integration of intermittent sources of energy by providing more reliable predictions and control of energy generation from such sources [Milligan et al. 

Q9. How long has the issue of integrating renewable energy supplies into the grid been studied?

The issue of integrating renewable energy supplies into the grid to reduce carbon emissions while ensuring their intermittency does not have an adverse impact (e.g., by requiring more spinning reserves or causing brownouts) has been actively researched for a number of years. 

Q10. How many agents are required to communicate their individual plans to the system?

their approach requires agents to communicate their individual plans (to optimise their resources) to the system, leading it to be scalable no more than ten thousand agents. 

Q11. How does the system perform when the supplier slightly overstates?

When the supplier slightly overstates γ, the authors can clearly see that cost increases exponentially (see overstating Figure 9(b)) while efficiency only increases by up to a negligible 0.5% (see overstating Figure 9(a)). 

Q12. What is the main problem with relying on price signals?

in general, relying solely on the price signal may result in a mismatch between the production capacity of the supplier and the demand from the consumers that might require the supplier to buy extra capacity at a high price or sell its extra production at a loss on the wholesale electricity market. 

Q13. What was the first time the idea of homeostatic control was pioneered?

While the idea of homeostatic control in this domain was pioneered more than 30 years ago, it was not deemed practical at the time due to the lack of communication and distributed computing technologies that would allow large populations of consumers of different types (of different income and usage levels) to participate in such mechanisms. 

Q14. What is the way to limit the amount of energy an agent consumes?

Since γ t+1 i implies a factor change in the current day’s overall consumption la,ti , the best an agent can do is to try and bound the total amount of energy it consumes at i by γt+1i l a,t i . 

Q15. What is the intuition that the suppliers can pass on some of their costs to their consumers?

The intuition is that if suppliers can pass on some of their costs, due to carbon taxes, to their consumers, the latter will be incentivised to align their behaviour to the needs of the supplier. 

Q16. How does the system perform when the supplier overstates?

On the other hand, when the supplier understates γ, the cost increases while efficiency decreases, such that the system is worse off both in terms of cost and efficiency.