Classification is one of the most active research and application areas of neural networks. The literature is vast and growing. This paper summarizes some of the most important developments in neural network classification research. Specifically, the issues of posterior probability estimation, the link between neural and conventional classifiers, learning and generalization tradeoff in classification, the feature variable selection, as well as the effect of misclassification costs are examined. Our purpose is to provide a synthesis of the published research in this area and stimulate further research interests and efforts in the identified topics.

/pdf/neural-networks-for-classification-a-survey-573tgz3hdm.pdf

Neural networks for classification: a survey

Introduction to multi-layer feed-forward neural networks

Many neural network classifiers provide outputs which estimate Bayesian a posteriori probabilities. When the estimation is accurate, network outputs can be treated as probabilities and sum to one. Simple proofs show that Bayesian probabilities are estimated when desired network outputs are 1 of M (one output unity, all others zero) and a squared-error or cross-entropy cost function is used. Results of Monte Carlo simulations performed using multilayer perceptron (MLP) networks trained with backpropagation, radial basis function (RBF) networks, and high-order polynomial networks graphically demonstrate that network outputs provide good estimates of Bayesian probabilities. Estimation accuracy depends on network complexity, the amount of training data, and the degree to which training data reflect true likelihood distributions and a priori class probabilities. Interpretation of network outputs as Bayesian probabilities allows outputs from multiple networks to be combined for higher level decision making, sim...

Neural Network Classifiers Estimate Bayesian a posteriori Probabilities.

https://people.missouristate.edu/RandallSexton/tabu.pdf

Global optimization for artificial neural networks: A tabu search application

Although, the genetic algorithm (GA) has been shown to be a superior neural network (NN) training method on computer-generated problems, its performance — on real world classification data sets is untested. To gain confidence that this alternative training technique is suitable for classification problems, a collection of 10 benchmark real world data sets were used in an extensive Monte Carlo study that compares backpropagation (BP) with the GA for NN training. We find that the GA reliably outperforms the commonly used BP algorithm as an alternative NN training technique. While this does not prove that the GA will always dominate BP, this demonstrated reliability with real world problems enables managers to use NNs trained with GAs as decision support tools with a greater degree of confidence.

/pdf/reliable-classification-using-neural-networks-a-genetic-1dgr0wc4ch.pdf

Reliable classification using neural networks: a genetic algorithm and backpropagation comparison

A variation of the back-propagation algorithm is described, using a log-likelihood cost function. Appropriate choices of learning parameters are discussed. An example is given where the range of initial weights leading to proper convergence is increased, and the number of iterations required is significantly reduced.

Convergence of back-propagation in neural networks using a log-likelihood cost function

A novel approach to adapting the weights of single-layer perceptron based channel equalisers is proposed, using the log-likelihood cost function and the resulting algorithm. Developing equalisers with this approach allows simpler computation, faster convergence and lower estimation error in comparison with the traditional back-propagation algorithm.

M.J.J. Holt

Papers

Convergence of back-propagation in neural networks using a log-likelihood cost function

Log-likelihood adaptive algorithm in single-layer perceptron based channel equalisation