Metaheuristic design of feedforward neural networks

Evolutionary artificial neural networks by multi-dimensional particle swarm optimization

Deep neural networks have been shown to outperform classical machine learning algorithms in solving real-world problems. However, the most successful deep neural networks were handcrafted from scratch taking the problem domain knowledge into consideration. This approach often consumes very significant time and computational resources. In this work, we propose a novel algorithm based on particle swarm optimization (PSO), capable of fast convergence when compared with others evolutionary approaches, to automatically search for meaningful deep convolutional neural networks (CNNs) architectures for image classification tasks, named psoCNN. A novel directly encoding strategy and a velocity operator were devised allowing the optimization use of PSO with CNNs. Our experimental results show that psoCNN can quickly find good CNN architectures that achieve quality performance comparable to the state-of-the-art designs.

Particle swarm optimization of deep neural networks architectures for image classification

ContextEnsembles of learning machines and locality are considered two important topics for the next research frontier on Software Effort Estimation (SEE). ObjectivesWe aim at (1) evaluating whether existing automated ensembles of learning machines generally improve SEEs given by single learning machines and which of them would be more useful; (2) analysing the adequacy of different locality approaches; and getting insight on (3) how to improve SEE and (4) how to evaluate/choose machine learning (ML) models for SEE. MethodA principled experimental framework is used for the analysis and to provide insights that are not based simply on intuition or speculation. A comprehensive experimental study of several automated ensembles, single learning machines and locality approaches, which present features potentially beneficial for SEE, is performed. Additionally, an analysis of feature selection and regression trees (RTs), and an investigation of two tailored forms of combining ensembles and locality are performed to provide further insight on improving SEE. ResultsBagging ensembles of RTs show to perform well, being highly ranked in terms of performance across different data sets, being frequently among the best approaches for each data set and rarely performing considerably worse than the best approach for any data set. They are recommended over other learning machines should an organisation have no resources to perform experiments to chose a model. Even though RTs have been shown to be more reliable locality approaches, other approaches such as k-Means and k-Nearest Neighbours can also perform well, in particular for more heterogeneous data sets. ConclusionCombining the power of automated ensembles and locality can lead to competitive results in SEE. By analysing such approaches, we provide several insights that can be used by future research in the area.

/pdf/ensembles-and-locality-on2yhvohg5.pdf

Ensembles and locality

http://sclab.yonsei.ac.kr/publications/Papers/IJ/An%20improved%20swarm%20optimized%20functional%20link%20artificial%20neural%20network%20(ISO-FLANN)%20for%20classification.pdf

An improved swarm optimized functional link artificial neural network (ISO-FLANN) for classification

The optimization of architecture and weights of feed forward neural networks is a complex task of great importance in problems of supervised learning. In this work we analyze the use of the particle swarm optimization algorithm for the optimization of neural network architectures and weights aiming better generalization performances through the creation of a compromise between low architectural complexity and low training errors. For evaluating these algorithms we apply them to benchmark classification problems of the medical field. The results showed that a PSO-PSO based approach represents a valid alternative to optimize weights and architectures of MLP neural networks.

Particle Swarm Optimization of Neural Network Architectures andWeights

An ANN-based automatic classifier for power system disturbance waveforms was developed. Actual voltage waveforms were applied in the training process. Signals were processed in two steps: i) decomposition through wavelet transformation up to the 5th decomposition level; ii) the resultant wavelet coefficients are processed via PCA, reducing the input space of the classifier to a much lower dimension. The classification was carried out using a combination of six MLPs with different architectures: five representing the first to fifth-level details, and one representing the fifth-level approximation. The RPROP algorithm was applied for training the networks. Network combination was formed using random committee which builds an ensemble of randomized base classifiers. Experimental results with real data indicate that the random committee is clearly an effective way to improve disturbance classification accuracy when compared with the simple average and the individual models.

Combining Multiple Artificial Neural Networks Using Random Committee to Decide upon Electrical Disturbance Classification

Wind Power forecasting is extremely important to assist in planning and programming studies for the operation of wind power generation. Several studies have shown that the Brazilian wind potential can contribute significantly to the electricity supply, especially in the Northeast Brazil, where winds present an important feature of being complementary in relation to the flows of the San Francisco River. However, using wind power to generate electricity has some drawbacks, such as uncertainties in generation and some difficulty in planning and operation of the power system. This paper presents actual results of wind power forecasting for two parks in the region of northeastern Brazil with four different models. Models that perform power generation forecasting using the forecasted wind speeds and the wind power curve of the park are called Wind to Power (W2P) and models that perform power generation forecasting using the historical power generation of the park are called Power to Power (P2P). The models perform forecasting of wind power generation with 6 hours ahead, discretized by 10 minutes and with 5 days ahead, discretized by 30 minutes. Models that directly predict the wind power (P2P) got the best results. These models were more suitable for use in the power systems operation planning considering the wind parks analyzed in northeastern Brazil.

Forecasting models of wind power in Northeastern of Brazil

An ANN-based automatic classifier for power system disturbance waveforms was developed. Actual voltage waveforms were applied in the training process. Signals are processed in two steps: i) decomposition through wavelet transformation up to the 5th decomposition level; ii) the resultant wavelet coefficients are processed via PCA, reducing the input space of the classifier to a much lower dimension. The classification is carried out using a combination of 3 MLPs with different architectures. The RPROP algorithm is applied for training the networks. Network combination was formed and the final decision of the classifier corresponds to the combination output with the highest value. The results showed to be quite promising for five disturbance types tested so far: sags, swells, harmonics, oscillatory transients and interruptions, as well as in the particular case of no disturbance.

M.A. Carvalho

Papers

Particle Swarm Optimization of Neural Network Architectures andWeights

Combining Multiple Artificial Neural Networks Using Random Committee to Decide upon Electrical Disturbance Classification

Forecasting models of wind power in Northeastern of Brazil

Improving Disturbance Classification by Combining Multiple Artificial Neural Networks