Showing papers by "Junhua Zhao published in 2007"

A Framework for Electricity Price Spike Analysis With Advanced Data Mining Methods

Abstract: There are many techniques for electricity market price forecasting. However, most of them are designed for expected price analysis rather than price spike forecasting. An effective method of predicting the occurrence of spikes has not yet been observed in the literature so far. In this paper, a data mining-based approach is presented to give a reliable forecast of the occurrence of price spikes. Combined with the spike value prediction techniques developed by the same authors, the proposed approach aims at providing a comprehensive tool for price spike forecasting. In this paper, feature selection techniques are first described to identify the attributes relevant to the occurrence of spikes. A simple introduction to the classification techniques is given for completeness. Two algorithms-support vector machine and probability classifier-are chosen to be the spike occurrence predictors and are discussed in detail. Realistic market data are used to test the proposed model with promising results

A Data Mining Based NTL Analysis Method

Abstract: This paper presents a method of determining which type of data provides maximum accuracy with reference to non-technical loss analysis in the electricity distribution sector. The method is based on two popular classification algorithms, Naive Bayesian and Decision Tree. It involves extracting the patterns of customers' kWh consumption behaviour from historical data and arranging the data in various ways by averaging them yearly, monthly, weekly, and daily. Both techniques are used and compared. The intention is to ensure the acquisition of optimum results in developing representative load profiles to be used as the reference for non-technical loss analysis directed at detecting any significant activities that may contribute to such losses.

Electricity market price spike forecasting and decision making

Abstract: Forecasting price spikes is a timely issue for the deregulated electricity market. Traditional price forecasting techniques show poor performance in handling price spikes, which usually follow a pattern different from the prices under normal market conditions. Therefore, novel approaches are required to forecast both the occurrences and values of spikes. In this paper a comprehensive study is conducted to investigate the performance of several data mining techniques for spike forecasting. Another major contribution of this paper is that a novel approach is proposed to integrate the spike forecasting process with decision-making, and to provide a comprehensive risk management tool against spikes. This approach is based on the Naive Bayesian Classifier. The benefits/costs of possible decisions are considered in the spike forecasting process to achieve the maximum benefits from the decisions against price spikes. We give a comprehensive theoretical proof of the proposed Bayesian classifier with benefit maximisation (BCBM) approach, which empirically demonstrates its effectiveness by achieving promising experiment results on real market price datasets.

Mining complex power networks for blackout prevention

Abstract: Following the recent devastating blackouts in North America, UK and Italy, blackout prevention has attracted significant attention, though it is known as a notoriously difficult task. To prevent the blackout, it is essential to accurately predict the instable status of power network components. In the large-scale power network however, existing analysis tools fail to perform accurate and in-time prediction of component instability, because of the sophisticated structure of real-world power networks and the huge amount of system variables to be analyzed. To prevent the blackout, we need an accurate and efficient method that (a) can discover interesting features and patterns relevant to the blackout, from the highly complex structure and ten thousands of system variables of a power network, and (b) can give accurate and fast prediction of system instability whenever required, so that the network operator can take necessary actions in time. In this paper, we report our tool developed for power network instability prediction. The proposed method consists of two major stages. In the first stage,a novel type of patterns namely Local Correlation Network Pattern (LCNP) is mined from the structure and system variables of the power network. Correlation rules, which are useful for the network operator to locate potentially instable components, can be further generated from the LCNP. In the second stage, a kernel based network classification method is developed to predict the system instability. By testing on a real world power network (the New England system), we demonstrate that the proposed tool is effective in predicting system instability and thus highly useful for blackout prevention.

Online rare events detection

Abstract: Rare events detection is regarded as an imbalanced classification problem, which attempts to detect the events with high impact but low probability. Rare events detection has many applications such as network intrusion detection and credit fraud detection. In this paper we propose a novel online algorithm for rare events detection. Different from traditional accuracy-oriented approaches, our approach employs a number of hypothesis tests to perform the cost/benefit analysis. Our approach can handle online data with unbounded data volume by setting up a proper moving-window size and a forgetting factor. A comprehensive theoretical proof of our algorithm is given. We also conduct the experiments that achieve significant improvements compared with the most relevant algorithms based on publicly available real-world datasets.

Developing GENCO's Strategic Bidding in an Electricity Market with Incomplete Information

Abstract: In deregulated electricity markets, market players have an important task of implementing the optimal offers, or bids, for each trading interval to maximize their profits. The major challenge of designing bidding strategies lies in that, it is difficult for a generator to predict competitive generators' behaviours because it only has incomplete information about its rivals. A novel approach of designing the optimal bidding strategies based on incomplete market information is proposed in this paper. This method predicts the expected bidding productions of each rival generator in the market based on publicly available bidding data. Moreover, the non-linear relationship between generators' bidding productions and the market clearing price (MCP) is also estimated from historical bidding and price data, using support vector machine (SVM). The optimal bidding problem is finally transformed into a stochastic optimization problem, which is solved with differential evolution (DE) and Monte Carlo simulation based on the predicted rivals' behaviour and MCP. The case studies using eleven coal-fired generators in the Australian National Electricity Market (NEM) are conducted to verify the effectiveness of the proposed method.

An Effective Approach to Predicting Electricity Market Price Spikes

Abstract: Electricity market price prediction is important for market participants. The most of the predicting techniques are designed for normal price predictions other than price spikes predictions. The aim of this paper is to analyse electricity market data including demand, price, and capacity reserve, to find out their causes to the occurrence of price spikes. The challenge of spike prediction is the accuracy of the prediction that is on how a classifier can capture all spikes that would happen. Particularly precision/recall is used in the evaluation of the spike prediction. It has shown that ELM (Extreme Learning Machine) algorithm has a superior performance in prediction of price spikes compared with other existing classification algorithms such as SVM (Support Vector Machine). The experiments and the evaluation of the results have confirmed these findings.

Electricity market management and analysis using advanced data mining and statistical methods

Abstract: The deregulated electricity markets have been in operation m a number ofncountries since the 1990s. During the deregulation process, vertically integratednpower utilities have been reformed into competitive markets, with initial goals tonimprove the market efficiency, minimize the production cost and reduce the electricitynprice. Given the benefits that have been achieved by the deregulation, several newnchallenges are also observed in the market. Due to the fundamental changes of thenelectric power industry, traditional management and analysis methods cannot dealnwith these new challenges. Novel electricity market management and analysisnmethods are therefore needed in the deregulated environment.nnnnnnn Data mining is defined as qthe nontrivial extraction of implicit, previouslynunknown, and potentially useful information from dataq and qthe science of extractingnuseful information from large data setsq. The modern electricity market producesnhuge amounts of market data, in which highly useful information can be extracted tonfacilitate the market management and analysis. In this thesis, I employ data miningntogether with advanced statistical methods to analyze the data of electricity markets.nThe data mining and statistical methods are integrated with the market managementnand analysis techniques to solve several difficult problems in electricity marketnresearch.nnnnnnnn This research aims at developing novel methods to solve several notoriouslyndifficult problems in the deregulated electricity market. The thesis consists of twonmain parts. The first part deals with extreme price volatility in the electricity market,nand the second part studies power system contingency assessment and prediction innthe deregulated market environment.nnnnnnn In the deregulated electricity market, extreme price volatility, which is also known as the price spike, is one of the major challenges not yet solved. Given their significant influences to market participants, price spike forecast together with normalnprice prediction are highly important in a competitive electricity market for individualnmarket participants as well as the system operator. In the first part of this thesis, annovel framework is proposed to handle the extreme price volatility caused bynelectricity price spikes. The framework is based on data mining and computationalnstatistics, thus is able to process the large data amount of electricity price signals. Innthe framework, feature selection techniques are used to identify relevant factors ofnprice spikes. Classification methods are employed to predict the occurrences of pricenspikes in the future. Based on the results of spike occurrence prediction, regressionnand time series models are used to forecast the value of the spike. In addition, anSupport Vector Machine (SVM) based forecasting model is proposed to estimate thenrisks involved in price spikes. I also develop a novel approach, namely BayesiannClassifier with Benefit Maximization (BCBM). The BCBM approach integrates thenprice spike prediction together with decision making of market participants, so as tonachieve the maximum decision benefits facing spikes.nnnnnnn In addition to the energy market, which is a complex economical system, thenphysical power system behind the electricity market is an essential integrated part ofnthe overall market as well. In Chapter 7, the problem of power system contingencynassessment and prediction is studied. In the deregulated market, the power system isnoperating under more stressed condition with much more uncertainties in comparisonnto the past. Following the recent devastating blackouts in USA, UK and Russia, powernsystem stability analysis and contingency prediction has attracted significant attentionnfrom both the academic society and industry. In this thesis, a novel method developednfor power system contingency prediction is reported. The proposed method consists ofntwo major stages. In the first stage, a novel type of patterns namely Local CorrelationnNetwork Pattern (LCNP) is mined from the structure and system variables of thenpower system. Correlation rules, which are useful for the network operator to locatenpotentially instable components, can be further generated from the LCNP. In thensecond stage, a kernel based classification method is developed to predict the systemninstability. By testing on a real-world power network (the New England system), Indemonstrate that the proposed method is effective in predicting system contingencynand thus highly useful for blackout prevention.nnnnnnnn In summary, the major contributions of this thesis includes a price spikenforecasting framework, a comprehensive empirical study of feature selection mnelectricity price forecasting, a novel statistical method for estimating the risks ofnelectricity prices, a data mining based approach for making decisions on spikes, and andata mining based approach for power system contingency analysis. This research isnfinished with 14 publications in major international journals and conferences.n