Metaheuristic design of feedforward neural networks

TL;DR: A broad spectrum of FNN optimization methodologies including conventional and metaheuristic approaches are summarized, which provides interesting research challenges for future research to cope-up with the present information processing era.

About: This article is published in Engineering Applications of Artificial Intelligence.The article was published on 2017-04-01 and is currently open access. It has received 398 citations till now. The article focuses on the topics: Metaheuristic & Extreme learning machine.

Figures

Figure 1: Three-layer feedforward neural network (a), where input layer has p input nodes, hidden layer has h activation functions, and output layer has q nodes.

Figure 6: Meta-learning scheme (a), where LR is learning parameter, ND is activation function, AR is architecture, and WT is weight [256]. Combined chromosome structure (b)

Figure 5: Mapping of phenotype (Fig. 4 to genotype (for architecture). (a) Direct encoding to a vector of connectivity matrix (Fig. 4(b)): i) upper right triangle; ii) complete connectivity. (b) Indirect encoding schemes for architecture (Fig. 4(a)), where S is a start symbol, A,B,C, and D are the variables, and a, c, i, and u is the terminal. (c) Complete connectivity derived from rules operation shown in Fig. 5(b).

Figure 3: Spectrum of metaheuristic design of FNN

Figure 4: Mapping of phenotype to genotype. (a) Phenotype of a three-layer FNN. (b) Adjacency matrix. (c) Weights encoding: i) real value; ii) 4-bits binary; iii) 4-bits gray encoding.

Figure 2: Metaheuristic may be used for finding initial weights WI2 and conventional algorithms may be for finding global optima P2 and vice versa [132].