The deep convolutional neural network (CNN) is the state-of-the-art solution for large-scale visual recognition. Following some basic principles such as increasing network depth and constructing highway connections, researchers have manually designed a lot of fixed network architectures and verified their effectiveness.,,In this paper, we discuss the possibility of learning deep network structures automatically. Note that the number of possible network structures increases exponentially with the number of layers in the network, which motivates us to adopt the genetic algorithm to efficiently explore this large search space. The core idea is to propose an encoding method to represent each network structure in a fixed-length binary string. The genetic algorithm is initialized by generating a set of randomized individuals. In each generation, we define standard genetic operations, e.g., selection, mutation and crossover, to generate competitive individuals and eliminate weak ones. The competitiveness of each individual is defined as its recognition accuracy, which is obtained via a standalone training process on a reference dataset. We run the genetic process on CIFAR10, a small-scale dataset, demonstrating its ability to find high-quality structures which are little studied before. The learned powerful structures are also transferrable to the ILSVRC2012 dataset for large-scale visual recognition.

Genetic CNN

Extreme learning machine (ELM) is a new learning algorithm for the single hidden layer feedforward neural networks Compared with the conventional neural network learning algorithm it overcomes the slow training speed and over-fitting problems ELM is based on empirical risk minimization theory and its learning process needs only a single iteration The algorithm avoids multiple iterations and local minimization It has been used in various fields and applications because of better generalization ability, robustness, and controllability and fast learning rate In this paper, we make a review of ELM latest research progress about the algorithms, theory and applications It first analyzes the theory and the algorithm ideas of ELM, then tracking describes the latest progress of ELM in recent years, including the model and specific applications of ELM, finally points out the research and development prospects of ELM in the future

Extreme learning machine: algorithm, theory and applications

Membrane systems (also called P systems) refer to the computing models abstracted from the structure and the functioning of the living cell as well as from the cooperation of cells in tissues, organs, and other populations of cells. Spiking neural P systems (SNPS) are a class of distributed and parallel computing models that incorporate the idea of spiking neurons into P systems. To attain the solution of optimization problems, P systems are used to properly organize evolutionary operators of heuristic approaches, which are named as membrane-inspired evolutionary algorithms (MIEAs). This paper proposes a novel way to design a P system for directly obtaining the approximate solutions of combinatorial optimization problems without the aid of evolutionary operators like in the case of MIEAs. To this aim, an extended spiking neural P system (ESNPS) has been proposed by introducing the probabilistic selection of evolution rules and multi-neurons output and a family of ESNPS, called optimization spiking neural P system (OSNPS), are further designed through introducing a guider to adaptively adjust rule probabilities to approximately solve combinatorial optimization problems. Extensive experiments on knapsack problems have been reported to experimentally prove the viability and effectiveness of the proposed neural system.

/pdf/an-optimization-spiking-neural-p-system-for-approximately-3a2dw9uok6.pdf

An optimization spiking neural p system for approximately solving combinatorial optimization problems.

This paper proposes a new hybrid stochastic training algorithm using the recently proposed grasshopper optimization algorithm (GOA) for multilayer perceptrons (MLPs) neural networks. The GOA algorithm is an emerging technique with a high potential in tackling optimization problems based on its flexible and adaptive searching mechanisms. It can demonstrate a satisfactory performance by escaping from local optima and balancing the exploration and exploitation trends. The proposed GOAMLP model is then applied to five important datasets: breast cancer, parkinson, diabetes, coronary heart disease, and orthopedic patients. The results are deeply validated in comparison with eight recent and well-regarded algorithms qualitatively and quantitatively. It is shown and proved that the proposed stochastic training algorithm GOAMLP is substantially beneficial in improving the classification rate of MLPs.

An efficient hybrid multilayer perceptron neural network with grasshopper optimization

Reservoirs are fundamental human-built infrastructures that collect, store, and deliver fresh surface water in a timely manner for many purposes. Efficient reservoir operation requires policy makers and operators to understand how reservoir inflows are changing under different hydrological and climatic conditions to enable forecast-informed operations. Over the last decade, the uses of Artificial Intelligence and Data Mining [AI & DM] techniques in assisting reservoir streamflow subseasonal to seasonal forecasts have been increasing. In this study, Random Forest [RF), Artificial Neural Network (ANN), and Support Vector Regression (SVR) are employed and compared with respect to their capabilities for predicting 1 month-ahead reservoir inflows for two headwater reservoirs in USA and China. Both current and lagged hydrological information and 17 known climate phenomenon indices, i.e., PDO and ENSO, etc., are selected as predictors for simulating reservoir inflows. Results show (1) three methods are capable of providing monthly reservoir inflows with satisfactory statistics; (2) the results obtained by Random Forest have the best statistical performances compared with the other two methods; (3) another advantage of Random Forest algorithm is its capability of interpreting raw model inputs; (4) climate phenomenon indices are useful in assisting monthly or seasonal forecasts of reservoir inflow; and (5) different climate conditions are autocorrelated with up to several months, and the climatic information and their lags are cross correlated with local hydrological conditions in our case studies.

/pdf/developing-reservoir-monthly-inflow-forecasts-using-8hpj8xc4p2.pdf

Developing reservoir monthly inflow forecasts using artificial intelligence and climate phenomenon information

This paper reviews the use of evolutionary algorithms (EAs) to optimize artificial neural networks (ANNs). First, we briefly introduce the basic principles of artificial neural networks and evolutionary algorithms and, by analyzing the advantages and disadvantages of EAs and ANNs, explain the advantages of using EAs to optimize ANNs. We then provide a brief survey on the basic theories and algorithms for optimizing the weights, optimizing the network architecture and optimizing the learning rules, and discuss recent research from these three aspects. Finally, we speculate on new trends in the development of this area.

Evolutionary artificial neural networks: a review

In research of pattern recognition, we always want to achieve the correct classification rate according to the characteristics required. Feature extraction greatly affects the design and performance of the classifier, and it is one of the core issue of PR research. As an important component of pattern recognition, feature extraction has been paid close attention by many scholars, and currently has become one of the research hot spots in the field of pattern recognition. This article gives a general discussion of feature extraction, includes linear feature extraction and nonlinear feature extraction, and introduces the frontier methods of this field, at last discusses the development tendency of feature extraction.

A survey on feature extraction for pattern recognition

In the traditional learning algorithms of radial basis function (RBF) neural network, the architecture of the network is hard to be decided; thereby, the learning ability and generalization ability are hard to achieve optimal. In this paper, we propose an algorithm to optimize the RBF neural network learning based on genetic algorithm; it uses hybrid encoding method, that is, encodes the network by binary encoding and encodes the weights by real encoding; the network architecture is self-adapted adjusted, and the weights are learned. Then, the network is further adjusted by pseudo inverse method or least mean square method. Experiments prove that the network gotten by this method has a better architecture and stronger classification ability, and the time of constructing the network artificially is saved. The algorithm is a self-adapted and intelligent learning algorithm.

An optimizing method of RBF neural network based on genetic algorithm

Back-Propagation (BP) neural network, as one of the most mature and most widespread algorithms, has the ability of large scale computing and has unique advantages when dealing with nonlinear high dimensional data. But when we manipulate high dimensional data with BP neural network, many feature variables provide enough information, but too many network inputs go against designing of the hidden-layer of the network and take up plenty of storage space as well as computing time, and in the process interfere the convergence of the training network, even influence the the accuracy of recognition finally. Factor analysis (FA) is a multivariate analysis method which transforms many feature variables into few synthetic variables. Aiming at the characteristics that the samples processed have more feature variables, combining with the structure feature of BP neural network, a FA-BP neural network algorithm is proposed. Firstly we reduce the dimensionality of the feature factor using FA, and then regard the features reduced as the input of the BP neural network, carry on network training and simulation with low dimensional data that we get. This algorithm here can simplify the network structure, improve the velocity of convergence, and save the running time. Then we apply the new algorithm in the field of pest prediction to emulate. The results show that under the prediction precision is not reduced, the error of the prediction value is reduced by using the new algorithm, and therefore the algorithm is effective.

An Improved BP Neural Network Algorithm Based on Factor Analysis.

Aiming at the large sample with high feature dimension, this paper proposes a back-propagation (BP) neural network algorithm based on factor analysis (FA) and cluster analysis (CA), which is combined with the principles of FA and CA, and the architecture of BP neural network. The new algorithm reduces the feature dimensionality of the initial data through FA to simplify the network architecture; then divides the samples into different sub-categories through CA, trains the network so as to improve the adaptability of the network. In application, it is first to classify the new samples, then using the corresponding network to predict. By an experiment, the new algorithm is significantly improved at the aspect of its prediction precision. In order to test and verify the validity of the new algorithm, we compare it with BP algorithms based on FA and CA.

Chunyang Su

Papers

Evolutionary artificial neural networks: a review

A survey on feature extraction for pattern recognition

An optimizing method of RBF neural network based on genetic algorithm

An Improved BP Neural Network Algorithm Based on Factor Analysis.

Research of neural network algorithm based on factor analysis and cluster analysis