Principles of QSAR models validation: internal and external
Reads0
Chats0
TLDR
Evidence is presented that only models that have been validated externally, after their internal validation, can be considered reliable and applicable for both external prediction and regulatory purposes.Abstract:
The recent REACH Policy of the European Union has led to scientists and regulators to focus their attention on establishing general validation principles for QSAR models in the context of chemical regulation (previously known as the Setubal, nowadays, the OECD principles). This paper gives a brief analysis of some principles: unambiguous algorithm, Applicability Domain (AD), and statistical validation. Some concerns related to QSAR algorithm reproducibility and an example of a fast check of the applicability domain for MLR models are presented. Common myths and misconceptions related to popular techniques for verifying internal predictivity, particularly for MLR models (for instance crossvalidation, bootstrap), are commented on and compared with commonly used statistical techniques for external validation. The differences in the two validating approaches are highlighted, and evidence is presented that only models that have been validated externally, after their internal validation, can be considered reliable and applicable for both external prediction and regulatory purposes. (“Validation is one of those words...that is constantly used and seldom defined” as stated by A. R. Feinstein in the book Multivariate Analysis: An Introduction, Yale University Press, New Haven, 1996).read more
Citations
More filters
Journal ArticleDOI
Best Practices for QSAR Model Development, Validation, and Exploitation.
TL;DR: Most critical QSAR modeling routines that are regarded as best practices in the field are examined, including procedures used to validate models, both internally and externally, as well as the need to define model applicability domains that should be used when models are employed for the prediction of external compounds or compound libraries.
Journal ArticleDOI
3D-QSAR in drug design--a review.
TL;DR: This review seeks to provide a bird's eye view of the different 3D-QSAR approaches employed within the current drug discovery community to construct predictive structure-activity relationships and discusses the limitations that are fundamental to these approaches, as well as those that might be overcome with the improved strategies.
Journal ArticleDOI
Deep learning for computational chemistry
TL;DR: Deep neural networks have been widely applied in the field of computational chemistry, including quantitative structure activity relationship, virtual screening, protein structure prediction, quantum chemistry, materials design, and property prediction as discussed by the authors.
Journal ArticleDOI
Real external predictivity of QSAR models: how to evaluate it? Comparison of different validation criteria and proposal of using the concordance correlation coefficient.
Nicola Chirico,Paola Gramatica +1 more
TL;DR: The concordance correlation coefficient is proposed as a complementary, or alternative, more prudent measure of a QSAR model to be externally predictive, and works well on real data sets, where it seems to be more stable, and helps in making decisions when the validation measures are in conflict.
Journal ArticleDOI
On a simple approach for determining applicability domain of QSAR models
TL;DR: The present study reports that the web application can be easily used for identification of the X-outliers for training set compounds and detection of the test compounds residing outside the AD using the descriptor pool of the training and test sets.
References
More filters
Journal ArticleDOI
Beware of q2
TL;DR: It is argued that the high value of LOO q2 appears to be the necessary but not the sufficient condition for the model to have a high predictive power, which is the general property of QSAR models developed using LOO cross-validation.
Journal ArticleDOI
The problem of overfitting.
TL;DR: The focus is on regression problems, which are those in which one of the measures, the dependent Variable, is of special interest, and the authors wish to explore its relationship with the other variables.
Journal ArticleDOI
The Importance of Being Earnest: Validation is the Absolute Essential for Successful Application and Interpretation of QSPR Models
TL;DR: A set of simple guidelines for developing validated and predictive QSPR models is presented, highlighting the need to establish the domain of model applicability in the chemical space to flag molecules for which predictions may be unreliable, and some algorithms that can be used for this purpose.
Journal ArticleDOI
Linear Model Selection by Cross-validation
TL;DR: In this article, the authors show that the inconsistency of the leave-one-out cross-validation can be rectified by using a leave-n v -out crossvalidation with n v, the number of observations reserved for validation, satisfying n v /n → 1 as n → ∞.
Journal ArticleDOI
Methods for reliability and uncertainty assessment and for applicability evaluations of classification- and regression-based QSARs.
Lennart Eriksson,Joanna Jaworska,Andrew Worth,Mark T. D. Cronin,Robert M McDowell,Paola Gramatica +5 more
TL;DR: This article provides an overview of methods for reliability assessment of quantitative structure-activity relationship (QSAR) models in the context of regulatory acceptance of human health and environmental QSARs.