Artificial Neural Networks (ANNs) are being used increasingly to predict and forecast water resources variables. In this paper, the steps that should be followed in the development of such models are outlined. These include the choice of performance criteria, the division and pre-processing of the available data, the determination of appropriate model inputs and network architecture, optimisation of the connection weights (training) and model validation. The options available to modellers at each of these steps are discussed and the issues that should be considered are highlighted. A review of 43 papers dealing with the use of neural network models for the prediction and forecasting of water resources variables is undertaken in terms of the modelling process adopted. In all but two of the papers reviewed, feedforward networks are used. The vast majority of these networks are trained using the backpropagation algorithm. Issues in relation to the optimal division of the available data, data pre-processing and the choice of appropriate model inputs are seldom considered. In addition, the process of choosing appropriate stopping criteria and optimising network geometry and internal network parameters is generally described poorly or carried out inadequately. All of the above factors can result in non-optimal model performance and an inability to draw meaningful comparisons between different models. Future research efforts should be directed towards the development of guidelines which assist with the development of ANN models and the choice of when ANNs should be used in preference to alternative approaches, the assessment of methods for extracting the knowledge that is contained in the connection weights of trained ANNs and the incorporation of uncertainty into ANN models.

Neural networks for the prediction and forecasting of water resources variables: a review of modelling issues and applications

This paper emphasizes the importance of rigorous validation as a crucial, integral component of Quantitative Structure Property Relationship (QSPR) model development. We consider some examples of published QSPR models, which in spite of their high fitted accuracy for the training sets and apparent mechanistic appeal, fail rigorous validation tests, and, thus, may lack practical utility as reliable screening tools. We present a set of simple guidelines for developing validated and predictive QSPR models. To this end, we discuss several validation strategies including (1) randomization of the modelled property, also called Y-scrambling, (2) multiple leave-many-out cross-validations, and (3) external validation using rational division of a dataset into training and test sets. We also highlight the need to establish the domain of model applicability in the chemical space to flag molecules for which predictions may be unreliable, and discuss some algorithms that can be used for this purpose. We advocate the broad use of these guidelines in the development of predictive QSPR models.

The Importance of Being Earnest: Validation is the Absolute Essential for Successful Application and Interpretation of QSPR Models

This article reviews some of the recent advances on FTIR spectroscopy in areas related to natural tissues and cell biology. It is the second review publication resulting from a detailed study on the applications of spectroscopic methods in biological studies and summarizes some of the most widely used peak frequencies and their assignments. The aim of these studies is to prepare a database of molecular fingerprints, which will help researchers in defining the chemical structure of the biological tissues introducing most of the important peaks present in the natural tissues. In spite of applying different methods, there seems to be a considerable similarity in defining the peaks of identical areas of the FTIR spectra. As a result, it is believed that preparing a unique collection of the frequencies encountered in FTIR spectroscopic studies can lead to significant improvements both in the quantity and quality of research and their outcomes. This article is the first review of its kind that provides...

Fourier Transform Infrared (FTIR) Spectroscopy of Biological Tissues

The characteristics of the OPLS method have been investigated for the purpose of discriminant analysis (OPLS-DA). We demonstrate how class-orthogonal variation can be exploited to augment classific ...

OPLS discriminant analysis: combining the strengths of PLS-DA and SIMCA classification†

This article provides an overview of methods for reliability assessment of quantitative structure-activity relationship (QSAR) models in the context of regulatory acceptance of human health and environmental QSARs. Useful diagnostic tools and data analytical approaches are highlighted and exemplified. Particular emphasis is given to the question of how to define the applicability borders of a QSAR and how to estimate parameter and prediction uncertainty. The article ends with a discussion regarding QSAR acceptability criteria. This discussion contains a list of recommended acceptability criteria, and we give reference values for important QSAR performance statistics. Finally, we emphasize that rigorous and independent validation of QSARs is an essential step toward their regulatory acceptance and implementation.

Methods for reliability and uncertainty assessment and for applicability evaluations of classification- and regression-based QSARs.

Bayesian regularized artificial neural networks (BRANNs) are more robust than standard back-propagation nets and can reduce or eliminate the need for lengthy cross-validation. Bayesian regularization is a mathematical process that converts a nonlinear regression into a "well-posed" statistical problem in the manner of a ridge regression. The advantage of BRANNs is that the models are robust and the validation process, which scales as O(N2) in normal regression methods, such as back propagation, is unnecessary. These networks provide solutions to a number of problems that arise in QSAR modeling, such as choice of model, robustness of model, choice of validation set, size of validation effort, and optimization of network architecture. They are difficult to overtrain, since evidence procedures provide an objective Bayesian criterion for stopping training. They are also difficult to overfit, because the BRANN calculates and trains on a number of effective network parameters or weights, effectively turning off those that are not relevant. This effective number is usually considerably smaller than the number of weights in a standard fully connected back-propagation neural net. Automatic relevance determination (ARD) of the input variables can be used with BRANNs, and this allows the network to "estimate" the importance of each input. The ARD method ensures that irrelevant or highly correlated indices used in the modeling are neglected as well as showing which are the most important variables for modeling the activity data. This chapter outlines the equations that define the BRANN method plus a flowchart for producing a BRANN-QSAR model. Some results of the use of BRANNs on a number of data sets are illustrated and compared with other linear and nonlinear models.

Bayesian regularization of neural networks.

Quantitative Structure Property Relationship Modeling of Diverse Materials Properties Tu Le, V. Chandana Epa, Frank R. Burden, and David A. Winkler* CSIRO Materials Science and Engineering, Bag 10, Clayton South MDC 3169, Australia CSIRO Materials Science and Engineering, 343 Royal Parade, Parkville 3052, Australia Monash Institute of Pharmaceutical Sciences, 381 Royal Parade, Parkville 3052, Australia

Quantitative Structure–Property Relationship Modeling of Diverse Materials Properties

A method for producing molecular identification numbers for hydrogen-depleted organic structures from the eigenvalues of a connectivity matrix is presented. Over 20000 structures have been successfully tested, and the method can also be used produce a unique numbering for the atoms in a structure and to identify which atoms belong to each of the substructures of a disconnected main structure

Molecular identification number for substructure searches

We describe the use of Bayesian regularized artificial neural networks (BRANNs) in the development of QSAR models. These networks have the potential to solve a number of problems which arise in QSAR modeling such as: choice of model; robustness of model; choice of validation set; size of validation effort; and optimization of network architecture. The application of the methods to QSAR of compounds active at the benzodiazepine and muscarinic receptors is illustrated.

Frank R. Burden

Papers

Bayesian regularization of neural networks.

Quantitative Structure–Property Relationship Modeling of Diverse Materials Properties

Molecular identification number for substructure searches

Robust QSAR models using Bayesian regularized neural networks.

Fourier Transform Infrared microspectroscopy and chemometrics as a tool for the discrimination of cyanobacterial strains