Artificial Neural Networks (ANNs) are being used increasingly to predict and forecast water resources variables. In this paper, the steps that should be followed in the development of such models are outlined. These include the choice of performance criteria, the division and pre-processing of the available data, the determination of appropriate model inputs and network architecture, optimisation of the connection weights (training) and model validation. The options available to modellers at each of these steps are discussed and the issues that should be considered are highlighted. A review of 43 papers dealing with the use of neural network models for the prediction and forecasting of water resources variables is undertaken in terms of the modelling process adopted. In all but two of the papers reviewed, feedforward networks are used. The vast majority of these networks are trained using the backpropagation algorithm. Issues in relation to the optimal division of the available data, data pre-processing and the choice of appropriate model inputs are seldom considered. In addition, the process of choosing appropriate stopping criteria and optimising network geometry and internal network parameters is generally described poorly or carried out inadequately. All of the above factors can result in non-optimal model performance and an inability to draw meaningful comparisons between different models. Future research efforts should be directed towards the development of guidelines which assist with the development of ANN models and the choice of when ANNs should be used in preference to alternative approaches, the assessment of methods for extracting the knowledge that is contained in the connection weights of trained ANNs and the incorporation of uncertainty into ANN models.

Neural networks for the prediction and forecasting of water resources variables: a review of modelling issues and applications

This paper emphasizes the importance of rigorous validation as a crucial, integral component of Quantitative Structure Property Relationship (QSPR) model development. We consider some examples of published QSPR models, which in spite of their high fitted accuracy for the training sets and apparent mechanistic appeal, fail rigorous validation tests, and, thus, may lack practical utility as reliable screening tools. We present a set of simple guidelines for developing validated and predictive QSPR models. To this end, we discuss several validation strategies including (1) randomization of the modelled property, also called Y-scrambling, (2) multiple leave-many-out cross-validations, and (3) external validation using rational division of a dataset into training and test sets. We also highlight the need to establish the domain of model applicability in the chemical space to flag molecules for which predictions may be unreliable, and discuss some algorithms that can be used for this purpose. We advocate the broad use of these guidelines in the development of predictive QSPR models.

The Importance of Being Earnest: Validation is the Absolute Essential for Successful Application and Interpretation of QSPR Models

This article reviews some of the recent advances on FTIR spectroscopy in areas related to natural tissues and cell biology. It is the second review publication resulting from a detailed study on the applications of spectroscopic methods in biological studies and summarizes some of the most widely used peak frequencies and their assignments. The aim of these studies is to prepare a database of molecular fingerprints, which will help researchers in defining the chemical structure of the biological tissues introducing most of the important peaks present in the natural tissues. In spite of applying different methods, there seems to be a considerable similarity in defining the peaks of identical areas of the FTIR spectra. As a result, it is believed that preparing a unique collection of the frequencies encountered in FTIR spectroscopic studies can lead to significant improvements both in the quantity and quality of research and their outcomes. This article is the first review of its kind that provides...

Fourier Transform Infrared (FTIR) Spectroscopy of Biological Tissues

The characteristics of the OPLS method have been investigated for the purpose of discriminant analysis (OPLS-DA). We demonstrate how class-orthogonal variation can be exploited to augment classific ...

OPLS discriminant analysis: combining the strengths of PLS-DA and SIMCA classification†

This article provides an overview of methods for reliability assessment of quantitative structure-activity relationship (QSAR) models in the context of regulatory acceptance of human health and environmental QSARs. Useful diagnostic tools and data analytical approaches are highlighted and exemplified. Particular emphasis is given to the question of how to define the applicability borders of a QSAR and how to estimate parameter and prediction uncertainty. The article ends with a discussion regarding QSAR acceptability criteria. This discussion contains a list of recommended acceptability criteria, and we give reference values for important QSAR performance statistics. Finally, we emphasize that rigorous and independent validation of QSARs is an essential step toward their regulatory acceptance and implementation.

Methods for reliability and uncertainty assessment and for applicability evaluations of classification- and regression-based QSARs.

Predicting maximum bioactivity by effective inversion of neural networks using genetic algorithms

Bayesian regularized artificial neural networks (BRANNs) are more robust than standard back-propagation nets and can reduce or eliminate the need for lengthy cross-validation. Bayesian regularization is a mathematical process that converts a nonlinear regression into a "well-posed" statistical problem in the manner of a ridge regression. The advantage of BRANNs is that the models are robust and the validation process, which scales as O(N2) in normal regression methods, such as back propagation, is unnecessary. These networks provide solutions to a number of problems that arise in QSAR modeling, such as choice of model, robustness of model, choice of validation set, size of validation effort, and optimization of network architecture. They are difficult to overtrain, since evidence procedures provide an objective Bayesian criterion for stopping training. They are also difficult to overfit, because the BRANN calculates and trains on a number of effective network parameters or weights, effectively turning off those that are not relevant. This effective number is usually considerably smaller than the number of weights in a standard fully connected back-propagation neural net. Automatic relevance determination (ARD) of the input variables can be used with BRANNs, and this allows the network to "estimate" the importance of each input. The ARD method ensures that irrelevant or highly correlated indices used in the modeling are neglected as well as showing which are the most important variables for modeling the activity data. This chapter outlines the equations that define the BRANN method plus a flowchart for producing a BRANN-QSAR model. Some results of the use of BRANNs on a number of data sets are illustrated and compared with other linear and nonlinear models.

Bayesian Regularization of Neural Networks.

Application of Neural Networks to Large Dataset QSAR, Virtual Screening, and Library Design

An oral dosage form is generally the most popular with patients. Many drug candidates fail in late development because of unfavourable absorption and pharmacokinetic profiles, or toxicity, among other factors (ADMET properties). This contributes to the fall in the efficiency of the pharmaceutical industry and to the rise in health costs. The ability to predict ADMET properties of drug leads can contribute to overcoming this problem. We have modelled intestinal absorption using several types of molecular descriptors and a non-linear Bayesian regularized neural network. Our models show very good predictive properties and are able to account for essentially all of the variance in the data that is not due to experimental error.

Predictive Human Intestinal Absorption QSAR Models Using Bayesian Regularized Neural Networks

Amphiphilic lyotropic liquid crystalline self-assembled nanomaterials have important applications in the delivery of therapeutic and imaging agents. However, little is known about the effect of the incorporated drug on the structure of nanoparticles. Predicting these properties is widely considered intractable. We present computational models for three drug delivery carriers, loaded with 10 drugs at six concentrations and two temperatures. These models predicted phase behavior for 11 new drugs. Subsequent synchrotron small-angle X-ray scattering experiments validated the predictions.

Frank R. Burden

Papers

Predicting maximum bioactivity by effective inversion of neural networks using genetic algorithms

Bayesian Regularization of Neural Networks.

Application of Neural Networks to Large Dataset QSAR, Virtual Screening, and Library Design

Predictive Human Intestinal Absorption QSAR Models Using Bayesian Regularized Neural Networks

Predicting the complex phase behavior of self-assembling drug delivery nanoparticles.