scispace - formally typeset
Topic

Verification and validation of computer simulation models

About: Verification and validation of computer simulation models is a(n) research topic. Over the lifetime, 1556 publication(s) have been published within this topic receiving 43203 citation(s).

...read more

Papers
More filters

Journal ArticleDOI
04 Feb 1994-Science
TL;DR: Verification and validation of numerical models of natural systems is impossible because natural systems are never closed and because model results are always nonunique.

...read more

Abstract: Verification and validation of numerical models of natural systems is impossible. This is because natural systems are never closed and because model results are always nonunique. Models can be confirmed by the demonstration of agreement between observation and prediction, but confirmation is inherently partial. Complete confirmation is logically precluded by the fallacy of affirming the consequent and by incomplete access to natural phenomena. Models can only be evaluated in relative terms, and their predictive value is always open to question. The primary value of models is heuristic.

...read more

2,757 citations


Posted Content
Jack P. C. Kleijnen1Institutions (1)
Abstract: This paper surveys verification and validation of models, especially simulation models in operations research. For verification it discusses 1) general good programming practice (such as modular programming), 2) checking intermediate simulation outputs through tracing and statistical testing per module, 3) statistical testing of final simulation outputs against analytical results, and 4) animation. For validation it discusses 1) obtaining real-worl data, 2) comparing simulated and real data through simple tests such as graphical, Schruben-Turing, and t tests, 3) testing whether simulated and real responses are positively correlated and moreover have the same mean, using two new statistical procedures based on regression analysis, 4) sensitivity analysis based on design of experiments and regression analysis, and risk or uncertainty analysis based on Monte Carlo sampling, and 5) white versus black box simulation models. Both verification and validation require good documentation, and are crucial parts of assessment, credibility, and accreditation. A bibliography with 61 references is included.

...read more

1,345 citations


Journal ArticleDOI
Robert G. Sargent1Institutions (1)
TL;DR: Three approaches to deciding model validity are described, two paradigms that relate verification and validation to the model development process are presented, and various validation techniques are defined.

...read more

Abstract: Verification and validation of simulation models are discussed in this paper. Three approaches to deciding model validity are described, two paradigms that relate verification and validation to the...

...read more

1,301 citations


Journal ArticleDOI
Abstract: Increasing concern over the implications of climate change for biodiversity has led to the use of species–climate envelope models to project species extinction risk under climatechange scenarios. However, recent studies have demonstrated significant variability in model predictions and there remains a pressing need to validate models and to reduce uncertainties. Model validation is problematic as predictions are made for events that have not yet occurred. Resubstituition and data partitioning of present-day data sets are, therefore, commonly used to test the predictive performance of models. However, these approaches suffer from the problems of spatial and temporal autocorrelation in the calibration and validation sets. Using observed distribution shifts among 116 British breeding-bird species over the past � 20 years, we are able to provide a first independent validation of four envelope modelling techniques under climate change. Results showed good to fair predictive performance on independent validation, although rules used to assess model performance are difficult to interpret in a decision-planning context. We also showed that measures of performance on nonindependent data provided optimistic estimates of models’ predictive ability on independent data. Artificial neural networks and generalized additive models provided generally more accurate predictions of species range shifts than generalized linear models or classification tree analysis. Data for independent model validation and replication of this study are rare and we argue that perfect validation may not in fact be conceptually possible. We also note that usefulness of models is contingent on both the questions being asked and the techniques used. Implementations of species–climate envelope models for testing hypotheses and predicting future events may prove wrong, while being potentially useful if put into appropriate context.

...read more

1,249 citations


5


Journal ArticleDOI
Edward J. Rykiel1Institutions (1)
TL;DR: The ecological literature reveals considerable confusion about the meaning of validation in the context of simulation models, and disagreements over the mean can only be resolved by establishing a convention.

...read more

Abstract: The ecological literature reveals considerable confusion about the meaning of validation in the context of simulation models. The confusion arises as much from semantic and philosophical considerations as from the selection of validation procedures. Validation is not a procedure for testing scientific theory or for certifying the ‘truth’ of current scientific understanding, nor is it a required activity of every modelling project. Validation means that a model is acceptable for its intended use because it meets specified performance requirements. Before validation is undertaken, (1) the purpose of the model, (2) the performance criteria, and (3) the model context must be specified. The validation process can be decomposed into several components: (1) operation, (2) theory, and (3) data. Important concepts needed to understand the model evaluation process are verification, calibration, validation, credibility, and qualification. These terms are defined in a limited technical sense applicable to the evaluation of simulation models, and not as general philosophical concepts. Different tests and standards are applied to the operational, theoretical, and data components. The operational and data components can be validated; the theoretical component cannot. The most common problem with ecological and environmental models is failure to state what the validation criteria are. Criteria must be explicitly stated because there are no universal standards for selecting what test procedures or criteria to use for validation. A test based on comparison of simulated versus observed data is generally included whenever possible. Because the objective and subjective components of validation are not mutually exclusive, disagreements over the meaning of validation can only be resolved by establishing a convention.

...read more

1,182 citations


Network Information
Related Topics (5)
Verification and validation

2.7K papers, 46.2K citations

83% related
Discrete event simulation

8.6K papers, 146.2K citations

82% related
System of systems

7.2K papers, 135.1K citations

81% related
Systems analysis

3.7K papers, 85.8K citations

80% related
Component-based software engineering

24.2K papers, 461.9K citations

79% related
Performance
Metrics
No. of papers in the topic in previous years
YearPapers
202115
20208
201923
201821
201742
201655

Top Attributes

Show by:

Topic's top 5 most impactful authors

Robert G. Sargent

19 papers, 1.6K citations

Megan Olsen

6 papers, 37 citations

Valeriy Vyatkin

5 papers, 169 citations

Nadia Hamani

4 papers, 7 citations

Anca I. Vermesan

4 papers, 101 citations