Machine Learning is the study of methods for programming computers to learn. Computers are applied to a wide range of tasks, and for most of these it is relatively easy for programmers to design and implement the necessary software. However, there are many tasks for which this is difficult or impossible. These can be divided into four general categories. First, there are problems for which there exist no human experts. For example, in modern automated manufacturing facilities, there is a need to predict machine failures before they occur by analyzing sensor readings. Because the machines are new, there are no human experts who can be interviewed by a programmer to provide the knowledge necessary to build a computer system. A machine learning system can study recorded data and subsequent machine failures and learn prediction rules. Second, there are problems where human experts exist, but where they are unable to explain their expertise. This is the case in many perceptual tasks, such as speech recognition, hand-writing recognition, and natural language understanding. Virtually all humans exhibit expert-level abilities on these tasks, but none of them can describe the detailed steps that they follow as they perform them. Fortunately, humans can provide machines with examples of the inputs and correct outputs for these tasks, so machine learning algorithms can learn to map the inputs to the outputs. Third, there are problems where phenomena are changing rapidly. In finance, for example, people would like to predict the future behavior of the stock market, of consumer purchases, or of exchange rates. These behaviors change frequently, so that even if a programmer could construct a good predictive computer program, it would need to be rewritten frequently. A learning program can relieve the programmer of this burden by constantly modifying and tuning a set of learned prediction rules. Fourth, there are applications that need to be customized for each computer user separately. Consider, for example, a program to filter unwanted electronic mail messages. Different users will need different filters. It is unreasonable to expect each user to program his or her own rules, and it is infeasible to provide every user with a software engineer to keep the rules up-to-date. A machine learning system can learn which mail messages the user rejects and maintain the filtering rules automatically. Machine learning addresses many of the same research questions as the fields of statistics, data mining, and psychology, but with differences of emphasis. Statistics focuses on understanding the phenomena that have generated the data, often with the goal of testing different hypotheses about those phenomena. Data mining seeks to find patterns in the data that are understandable by people. Psychological studies of human learning aspire to understand the mechanisms underlying the various learning behaviors exhibited by people (concept learning, skill acquisition, strategy change, etc.).

Machine learning

Forecasting with artificial neural networks: the state of the art

Learning and evolution are two fundamental forms of adaptation. There has been a great interest in combining learning and evolution with artificial neural networks (ANNs) in recent years. This paper: 1) reviews different combinations between ANNs and evolutionary algorithms (EAs), including using EAs to evolve ANN connection weights, architectures, learning rules, and input features; 2) discusses different search operators which have been used in various EAs; and 3) points out possible future research directions. It is shown, through a considerably large literature review, that combinations between ANNs and EAs can lead to significantly better intelligent systems than relying on ANNs or EAs alone.

/pdf/evolving-artificial-neural-networks-4kzrbu7pkn.pdf

Evolving artificial neural networks

A unified view of metaheuristics This book provides a complete background on metaheuristics and shows readers how to design and implement efficient algorithms to solve complex optimization problems across a diverse range of applications, from networking and bioinformatics to engineering design, routing, and scheduling. It presents the main design questions for all families of metaheuristics and clearly illustrates how to implement the algorithms under a software framework to reuse both the design and code. Throughout the book, the key search components of metaheuristics are considered as a toolbox for: Designing efficient metaheuristics (e.g. local search, tabu search, simulated annealing, evolutionary algorithms, particle swarm optimization, scatter search, ant colonies, bee colonies, artificial immune systems) for optimization problems Designing efficient metaheuristics for multi-objective optimization problems Designing hybrid, parallel, and distributed metaheuristics Implementing metaheuristics on sequential and parallel machines Using many case studies and treating design and implementation independently, this book gives readers the skills necessary to solve large-scale optimization problems quickly and efficiently. It is a valuable reference for practicing engineers and researchers from diverse areas dealing with optimization or machine learning; and graduate students in computer science, operations research, control, engineering, business and management, and applied mathematics.

/pdf/metaheuristics-from-design-to-implementation-1vqdfv8nk9.pdf

Metaheuristics: From Design to Implementation

We present a tree-structured architecture for supervised learning. The statistical model underlying the architecture is a hierarchical mixture model in which both the mixture coefficients and the mixture components are generalized linear models (GLIM's). Learning is treated as a maximum likelihood problem; in particular, we present an Expectation-Maximization (EM) algorithm for adjusting the parameters of the architecture. We also develop an on-line learning algorithm in which the parameters are updated incrementally. Comparative simulation results are presented in the robot dynamics domain.

/pdf/hierarchical-mixtures-of-experts-and-the-em-algorithm-21xcy88aw9.pdf

Hierarchical mixtures of experts and the EM algorithm

This paper presents a genetic algorithm (GA) solution to the unit commitment problem. GAs are general purpose optimization techniques based on principles inspired from the biological evolution using metaphors of mechanisms such as natural selection, genetic recombination and survival of the fittest. A simple GA algorithm implementation using the standard crossover and mutation operators could locate near optimal solutions but in most cases failed to converge to the optimal solution. However, using the varying quality function technique and adding problem specific operators, satisfactory solutions to the unit commitment problem were obtained. Test results for power systems of up to 100 units and comparisons with results obtained using Lagrangian relaxation and dynamic programming are also reported.

A genetic algorithm solution to the unit commitment problem

This paper presents an enhanced genetic algorithm (EGA) for the solution of the optimal power flow (OPF) with both continuous and discrete control variables. The continuous control variables modeled are unit active power outputs and generator-bus voltage magnitudes, while the discrete ones are transformer-tap settings and switchable shunt devices. A number of functional operating constraints, such as branch flow limits, load bus voltage magnitude limits, and generator reactive capabilities, are included as penalties in the GA fitness function (FF). Advanced and problem-specific operators are introduced in order to enhance the algorithm's efficiency and accuracy. Numerical results on two test systems are presented and compared with results of other approaches.

Optimal power flow by enhanced genetic algorithm

This paper presents the development of an artificial neural network (ANN) based short-term load forecasting model for the Energy Control Center of the Greek Public Power Corporation (PPC). The model can forecast daily load profiles with a lead time of one to seven days. Attention was paid for the accurate modeling of holidays. Experiences gained during the development of the model regarding the selection of the input variables, the ANN structure, and the training data set are described in the paper. The results indicate that the load forecasting model developed provides accurate forecasts.

A neural network short term load forecasting model for the Greek power system

Two genetic algorithm (GA) solutions to the economic dispatch problem are presented. An advantage of the GA solutions is that they do not impose any convexity restrictions on the generator cost functions. Another advantage is that GAs can be very effectively coded to work on parallel machines. Test results with systems of up to 72 generating units with nonconvex cost functions show that both GAs outperform the dynamic programming solution to the economic dispatch problem. Furthermore, the execution time of the second GA solution increases almost linearly with the number of generators.

Genetic algorithm solution to the economic dispatch problem

We investigate the potential of a microgenetic algorithm (MGA) as a generalized hill-climbing operator. Combining a standard GA with the suggested MGA operator leads to a hybrid genetic scheme GA-MGA, with enhanced searching qualities. The main GA performs global search while the MGA explores a neighborhood of the current solution provided by the main GA, looking for better solutions. The MGA operator performs genetic local search. The major advantage of MGA is its ability to identify and follow narrow ridges of arbitrary direction leading to the global optimum. The proposed GA-MGA scheme is tested against 13 different schemes, including a simple GA and GAs with different hill-climbing operators. Experiments are conducted on a test set including eight constrained optimization problems with continuous variables. Extensive simulation results demonstrate the efficiency of the proposed GA-MGA scheme. For the same number of fitness evaluations, GA-MGA exhibited a significantly better performance in terms of solution accuracy, feasibility percentage of the attained solutions, and robustness.

Vassilios Petridis

Papers

A genetic algorithm solution to the unit commitment problem

Optimal power flow by enhanced genetic algorithm

A neural network short term load forecasting model for the Greek power system

Genetic algorithm solution to the economic dispatch problem

Microgenetic algorithms as generalized hill-climbing operators for GA optimization