Journal Article•
Optimizing Weights of Artificial Neural Networks using Genetic Algorithms
28 Dec 2012-International Journal of Advanced Research in Computer Science and Electronics Engineering-Vol. 1, Iss: 10
TL;DR: Genetic algorithms are a class of optimization procedures which are well suited to the problem of training and optimize weights of Artificial Neural Networks and are shown in this paper.
Abstract: Artificial Neural Networks have a number of properties which make them psuitable to solve complex pattern classification problems. Their applications to some real world problems has been adopted by the lack of a training algorithm. This algorithms finds a nearly globally optimal set of weights in a relatively short time. Back propagation is one of the training algorithm of the Artificial neural network. However, training the neural networks using backpropagation algorithm may cause two main drawbacks: trapping into local minima and converging slowly. In view of these limitations of back-propagation neural networks, global search technique such as Genetic algorithm have been presented to overcome these shortcomings. Genetic algorithms are a class of optimization procedures which are good at exploring a large and complex space in an intelligent way. It finds values close to the global optimum. Hence, they are well suited to the problem of training and optimize weights of Artificial Neural Networks. In this paper the use of Genetic algorithms to optimize weights of Artificial Neural Networks is shown.
Citations
More filters
TL;DR: Results indicate that implementation of GA-based ANN models as a highly-reliable, efficient and practical tool in predicting the pile bearing capacity is of advantage.
Abstract: The application of artificial neural network (ANN) in predicting pile bearing capacity is underlined in several studies. However, ANN deficiencies in finding global minima as well as its slow rate of convergence are the major drawbacks of implementing this technique. The current study aimed at developing an ANN-based predictive model enhanced with genetic algorithm (GA) optimization technique to predict the bearing capacity of piles. To provide necessary dataset required for establishing the model, 50 dynamic load tests were conducted on precast concrete piles in Pekanbaru, Indonesia. The pile geometrical properties, pile set, hammer weight and drop height were set to be the network inputs and the pile ultimate bearing capacity was set to be the output of the GA-based ANN model. The best predictive model was selected after conducting a sensitivity analysis for determining the optimum GA parameters coupled with a trial-and-error method for finding the optimum network architecture i.e. number of hidden nodes. Results indicate that the pile bearing capacities predicted by GA-based ANN are in close agreement with measured bearing capacities. Coefficient of determination as well as mean square error equal to 0.990 and 0.002 for testing datasets respectively, suggest that implementation of GA-based ANN models as a highly-reliable, efficient and practical tool in predicting the pile bearing capacity is of advantage.
270 citations
Cites methods from "Optimizing Weights of Artificial Ne..."
...An inverse procedure is repeated for forming the second offspring [41]....
[...]
TL;DR: Comparison between the coefficients of determination, R2, obtained through conventional ANN and PSO-based ANN techniques reveal the superiority of the PSO -based ANN model in predicting UCS.
Abstract: Many attempts have been made to predict unconfined compressive strength (UCS) of rocks using back-propagation (BP) artificial neural network (ANN). However, BP-ANN suffers from some disadvantages such as slow rate of learning and getting trapped in local minima. Utilization of particle swarm optimization (PSO) algorithm as a mechanism to improve the performance of ANNs is recently underlined in literature. The objective of this paper is to develop a PSO-based ANN predictive model of UCS. For this reason, a comprehensive experimental program was conducted on 66 granite and limestone sample sets taken from different states in Malaysia. The experimental program consists of direct and indirect estimation of UCS of rocks. The results of laboratory tests including point load index test (IS(50)), Schmidt hammer rebound number (SRn), p-wave velocity test (Vp) and dry density (DD) test were used as inputs of the network while UCS results were set to be the output. For comparison purpose, the prediction performance of the proposed hybrid model was checked against that of a conventional ANN. Comparison between the coefficients of determination, R2, obtained through conventional ANN and PSO-based ANN techniques reveal the superiority of the PSO-based ANN model in predicting UCS. In overall, the R2 for the proposed hybrid predictive model was 0.97 while in case of conventional ANN, the R2 was found to be 0.71. By performing sensitivity analysis, it was concluded that the effect of DD and SRn on predicted UCS values is slightly higher compared to other parameters.
255 citations
TL;DR: Empirical results show that the Type 2 input variables can generate a higher forecast accuracy and that it is possible to enhance the performance of the optimized ANN model by selecting input variables appropriately.
Abstract: In the business sector, it has always been a difficult task to predict the exact daily price of the stock market index; hence, there is a great deal of research being conducted regarding the prediction of the direction of stock price index movement. Many factors such as political events, general economic conditions, and traders’ expectations may have an influence on the stock market index. There are numerous research studies that use similar indicators to forecast the direction of the stock market index. In this study, we compare two basic types of input variables to predict the direction of the daily stock market index. The main contribution of this study is the ability to predict the direction of the next day’s price of the Japanese stock market index by using an optimized artificial neural network (ANN) model. To improve the prediction accuracy of the trend of the stock market index in the future, we optimize the ANN model using genetic algorithms (GA). We demonstrate and verify the predictability of stock price direction by using the hybrid GA-ANN model and then compare the performance with prior studies. Empirical results show that the Type 2 input variables can generate a higher forecast accuracy and that it is possible to enhance the performance of the optimized ANN model by selecting input variables appropriately.
198 citations
Cites result from "Optimizing Weights of Artificial Ne..."
...The results of these studies support the notion that GA can enhance the accuracy of ANNmodels and can decrease the time required for experimentation [35]....
[...]
TL;DR: An artificial neural network that can map any nonlinear function without a prior assumption to predict the return of the Japanese Nikkei 225 index is applied and it is observed through empirical experiments that the selected input variables were effective to predict stock market returns.
Abstract: Accurate prediction of stock market returns is a very challenging task because of the highly nonlinear nature of the financial time series. In this study, we apply an artificial neural network (ANN) that can map any nonlinear function without a prior assumption to predict the return of the Japanese Nikkei 225 index. (1) To improve the effectiveness of prediction algorithms, we propose a new set of input variables for ANN models. (2) To verify the prediction ability of the selected input variables, we predict returns for the Nikkei 225 index using the classical back propagation (BP) learning algorithm. (3) Global search techniques, i.e., a genetic algorithm (GA) and simulated annealing (SA), are employed to improve the prediction accuracy of the ANN and overcome the local convergence problem of the BP algorithm. It is observed through empirical experiments that the selected input variables were effective to predict stock market returns. A hybrid approach based on GA and SA improve prediction accuracy significantly and outperform the traditional BP training algorithm.
172 citations
TL;DR: Investigation of the application of a hybrid artificial neural network–particle swarm optimization (ANN-PSO) model in the behavior prediction of channel connectors embedded in normal and high-strength concrete (HSC) revealed that an ANN model could properly predict the behavior of channel connector and eliminate the need for conducting costly experiments to some extent.
Abstract: Channel shear connectors are known as an appropriate alternative for common shear connectors due to having a lower manufacturing cost and an easier installation process. The behavior of channel connectors is generally determined through conducting experiments. However, these experiments are not only costly but also time-consuming. Moreover, the impact of other parameters cannot be easily seen in the behavior of the connectors. This paper aims to investigate the application of a hybrid artificial neural network–particle swarm optimization (ANN-PSO) model in the behavior prediction of channel connectors embedded in normal and high-strength concrete (HSC). To generate the required data, an experimental project was conducted. Dimensions of the channel connectors and the compressive strength of concrete were adopted as the inputs of the model, and load and slip were predicted as the outputs. To evaluate the ANN-PSO model, an ANN model was also developed and tuned by a backpropagation (BP) learning algorithm. The results of the paper revealed that an ANN model could properly predict the behavior of channel connectors and eliminate the need for conducting costly experiments to some extent. In addition, in this case, the ANN-PSO model showed better performance than the ANN-BP model by resulting in superior performance indices.
166 citations
References
More filters
Proceedings Article•
20 Aug 1989
TL;DR: A set of experiments performed on data from a sonar image classification problem are described to illustrate the improvements gained by using a genetic algorithm rather than backpropagation and chronicle the evolution of the performance of the genetic algorithm as it added more and more domain-specific knowledge into it.
Abstract: Multilayered feedforward neural networks possess a number of properties which make them particularly suited to complex pattern classification problems. However, their application to some realworld problems has been hampered by the lack of a training algonthm which reliably finds a nearly globally optimal set of weights in a relatively short time. Genetic algorithms are a class of optimization procedures which are good at exploring a large and complex space in an intelligent way to find values close to the global optimum. Hence, they are well suited to the problem of training feedforward networks. In this paper, we describe a set of experiments performed on data from a sonar image classification problem. These experiments both 1) illustrate the improvements gained by using a genetic algorithm rather than backpropagation and 2) chronicle the evolution of the performance of the genetic algorithm as we added more and more domain-specific knowledge into it.
1,087 citations
TL;DR: A brief tutorial on sequence learning is presented, which requires comparing, contrasting, and combining the existing techniques, approaches, and paradigms, to develop better, more powerful algorithms.
Abstract: 2 1094-7167/01/$10.00 © 2001 IEEE IEEE INTELLIGENT SYSTEMS So, it’s logical that sequence learning is an important component of learning in many task domains of intelligent systems: inference, planning, reasoning, robotics, natural language processing, speech recognition, adaptive control, time series prediction, financial engineering, DNA sequencing, and so on. Naturally, the unique perspectives of these domains lead to different sequence-learning approaches. These approaches deal with somewhat differently formulated sequence-learning problems (for example, some with actions and some without) and with different aspects of sequence learning (for example, sequence prediction versus sequence recognition). Despite the plethora of approaches, sequence learning is still difficult. We believe that the right approach to improving sequence learning is to first better understand the state of the art in the different disciplines related to this topic. This requires comparing, contrasting, and combining the existing techniques, approaches, and paradigms, to develop better, more powerful algorithms. Toward that end, we present here a brief tutorial on sequence learning.
151 citations
08 Jul 1991
TL;DR: The probability of incorrectly saturated output nodes at the beginning epoch of learning is derived as a function of the range of initial weights, the number of nodes in each layer, and the maximum slope of the sigmoidal activation function.
Abstract: The critical drawback of the backpropagation learning algorithm is its slow error convergence. The major reason for this is the premature saturation, a phenomenon in which the error of a neural network stays almost constant for some period of time during learning. It is known to be caused by an inappropriate set of initial weights. The probability of incorrectly saturated output nodes at the beginning epoch of learning is derived as a function of the range of initial weights, the number of nodes in each layer, and the maximum slope of the sigmoidal activation function. This is verified by Monte Carlo simulation. >
108 citations
TL;DR: The preliminary study for only using simple GA has been proved to be effective for improving the accuracy of artificial neural networks.
Abstract: This paper aims to find the optimal set of initial weights to enhance the accuracy of artificial neural networks (ANNs) by using genetic algorithms (GA). The sample in this study included 228 patients with first low-trauma hip fracture and 215 patients without hip fracture, both of them were interviewed with 78 questions. We used logistic regression to select 5 important factors (i.e., bone mineral density, experience of fracture, average hand grip strength, intake of coffee, and peak expiratory flow rate) for building artificial neural networks to predict the probabilities of hip fractures. Three-layer (one hidden layer) ANNs models with back-propagation training algorithms were adopted. The purpose in this paper is to find the optimal initial weights of neural networks via genetic algorithm to improve the predictability. Area under the ROC curve (AUC) was used to assess the performance of neural networks. The study results showed the genetic algorithm obtained an AUC of 0.858 ± 0.00493 on modeling data and 0.802±0.03318 on testing data. They were slightly better than the results of our previous study (0.868±0.00387 and 0.796±0.02559, resp.). Thus, the preliminary study for only using simple GA has been proved to be effective for improving the accuracy of artificial neural networks.
48 citations