Testing ecological models: the meaning of validation
TL;DR: The ecological literature reveals considerable confusion about the meaning of validation in the context of simulation models, and disagreements over the mean can only be resolved by establishing a convention.
About: This article is published in Ecological Modelling.The article was published on 1996-11-01. It has received 1238 citations till now. The article focuses on the topics: Validation rule & Data validation.
Citations
More filters
TL;DR: A review of predictive habitat distribution modeling is presented, which shows that a wide array of models has been developed to cover aspects as diverse as biogeography, conservation biology, climate change research, and habitat or species management.
6,748 citations
TL;DR: The SWAT-CUP tool as discussed by the authors is a semi-distributed river basin model that requires a large number of input parameters, which complicates model parameterization and calibration, and is used to provide statistics for goodness-of-fit.
Abstract: SWAT (Soil and Water Assessment Tool) is a comprehensive, semi-distributed river basin model that requires a large number of input parameters, which complicates model parameterization and calibration. Several calibration techniques have been developed for SWAT, including manual calibration procedures and automated procedures using the shuffled complex evolution method and other common methods. In addition, SWAT-CUP was recently developed and provides a decision-making framework that incorporates a semi-automated approach (SUFI2) using both manual and automated calibration and incorporating sensitivity and uncertainty analysis. In SWAT-CUP, users can manually adjust parameters and ranges iteratively between autocalibration runs. Parameter sensitivity analysis helps focus the calibration and uncertainty analysis and is used to provide statistics for goodness-of-fit. The user interaction or manual component of the SWAT-CUP calibration forces the user to obtain a better understanding of the overall hydrologic processes (e.g., baseflow ratios, ET, sediment sources and sinks, crop yields, and nutrient balances) and of parameter sensitivity. It is important for future calibration developments to spatially account for hydrologic processes; improve model run time efficiency; include the impact of uncertainty in the conceptual model, model parameters, and measured variables used in calibration; and assist users in checking for model errors. When calibrating a physically based model like SWAT, it is important to remember that all model input parameters must be kept within a realistic uncertainty range and that no automatic procedure can substitute for actual physical knowledge of the watershed.
2,200 citations
TL;DR: A series of papers prepared within the framework of an international workshop entitled: Advances in GLMs /GAMs modeling: from species distribution to environmental management, held in Riederalp, Switzerland, 6 � /11 August 2001 are introduced.
2,006 citations
Cites background from "Testing ecological models: the mean..."
...Even fewer perform field validation ( Rykiel, 1996; Manel et al., 2002), calling into question the ultimate validity and application of the models (Guisan and Zimmermann, 2000)....
[...]
...Even fewer perform field validation (Rykiel 1996; Manel et al. in press), calling into question the ultimate validity and application of the models (Guisan & Zimmermann 2000)....
[...]
TL;DR: Five areas of enquiry are identified and discussed that are of high importance for species distribution modelling: clarification of the niche concept; improved designs for sampling data for building models; improved parameterization; improved model selection and predictor contribution; and improved model evaluation.
Abstract: Species distribution modelling is central to both fundamental and applied research in biogeography. Despite widespread use of models, there are still important conceptual ambiguities as well as biotic and algorithmic uncertainties that need to be investigated in order to increase confidence in model results. We identify and discuss five areas of enquiry that are of high importance for species distribution modelling: (1) clarification of the niche concept; (2) improved designs for sampling data for building models; (3) improved parameterization; (4) improved model selection and predictor contribution; and (5) improved model evaluation. The challenges discussed in this essay do not preclude the need for developments of other areas of research in this field. However, they are critical for allowing the science of species distribution modelling to move forward.
1,649 citations
Additional excerpts
...CSIC, 28006, Madrid, Spain, 2Centre for Macroecology, Institute of Biology, Universitetsparken 15, DK-2100 Copenhagen, Denmark, 3Department of Ecology and...
[...]
Book•
01 Jan 2005
TL;DR: An excellent introduction and overview of this field, written by Volker Grimm and Steven F. Railsback, should be read by everyone interested in individual-based modeling and especially by anyone contemplating developing, or being involved with a group developing, an individualbased model.
Abstract: Individual-based modeling is a new, exciting discipline that allows ecologists to explore, using computer simulations, how properties of populations and ecosystems might evolve from the characteristics and behaviors of individual organisms. Individual-based Modeling and Ecology, written by Volker Grimm and Steven F. Railsback, gives an excellent introduction and overview of this field. It should be read by everyone interested in individual-based modeling, and especially by anyone contemplating developing, or being involved with a group developing, an individualbased model.
1,495 citations
References
More filters
Book•
01 Jan 1982
TL;DR: The text is designed for a one-term or two-quarter course in simulation offered in departments of industrial engineering, business, computer science and operations research.
Abstract: From the Publisher:
This second edition of Simulation Modeling and Analysis includes a chapter on "Simulation in Manufacturing Systems" and examples. The text is designed for a one-term or two-quarter course in simulation offered in departments of industrial engineering,business,computer science and operations research.
9,905 citations
"Testing ecological models: the mean..." refers background in this paper
...Some authors suggest that models can be validated (Law and Kelton, 1991), while others contend that models can only be invalidated (e....
[...]
TL;DR: In this article, a model of soil organic matter (SOM) quantity and composition was used to simulate steady-state organic matter levels for 24 grassland locations in the U.S. Great Plains.
Abstract: We analyzed climatic and textural controls of soil organic C and N for soils of the U.S. Great Plains. We used a model of soil organic matter (SOM) quantity and composition to simulate steady-state organic matter levels for 24 grassland locations in the Great Plains. The model was able to simulate the effects of climatic gradients on SOM and productivity. Soil texture was also a major control over organic matter dynamics. The model adequately predicted above-ground plant production and soil C and N levels across soil textures (sandy, medium, and fine); however, the model tended to overestimate soil C and N levels for fine textured soil by 10 to 15%. The impact of grazing on the system was simulated and showed that steady-state soil C and N levels were sensitive to the grazing intensity, with soil C and N levels decreasing with increased grazing rates. Regional trends in SOM can be predicted using four site-specific variables, temperature, moisture, soil texture, and plant lignin content. Nitrogen inputs must also be known. Grazing intensity during soil development is also a significant control over steady-state levels of SOM, and since few data are available on presettlement grazing, some uncertainty is inherent in the model predictions
3,594 citations
"Testing ecological models: the mean..." refers methods in this paper
...For operational validation, the demonstration involves a comparison of simulated data with data obtained by observation and measurement of the real system (e.g., Parton et al., 1987; Mayer and Butler, 1993)....
[...]
Book•
01 Sep 2005
TL;DR: In this article, various methods of environmental impact assessment as a guide to design of new environmental development and management projects are discussed. But the authors do not reject the concept of the environmental impact analysis but rather stress the need for fundamental understanding of the structure and dynamics of ecosystems.
Abstract: This book is on the various methods of environmental impact assessment as a guide to design of new environmental development and management projects. This approach surveys the features of the environment likely to be affected by the developments under consideration, analyses the information collected, tries to predict the impact of these developments and lays down guidelines or rules for their management.
This book is concerned with practical problems, e.g. development in Canada, the management of fisheries, pest control, etc. It is devoted to a general understanding of environmental systems through methods that have worked in the real world with its many uncertainties. It does not reject the concept of environmental impact analysis but rather stresses the need for fundamental understanding of the structure and dynamics of ecosystems.
3,437 citations
TL;DR: Verification and validation of numerical models of natural systems is impossible because natural systems are never closed and because model results are always nonunique.
Abstract: Verification and validation of numerical models of natural systems is impossible. This is because natural systems are never closed and because model results are always nonunique. Models can be confirmed by the demonstration of agreement between observation and prediction, but confirmation is inherently partial. Complete confirmation is logically precluded by the fallacy of affirming the consequent and by incomplete access to natural phenomena. Models can only be evaluated in relative terms, and their predictive value is always open to question. The primary value of models is heuristic.
2,909 citations
"Testing ecological models: the mean..." refers background in this paper
...Such a test cannot demonstrate the logical validity of the model’s scientific content (Oreskes et al., 1994)....
[...]
...The terms confirmation and corroboration have been proposed as alternatives (Swartzman and Kaluzny, 1987; Oreskes et al., 1994)....
[...]
...Validation means that a model is acceptable for its intended use because it meets specified performance requirements....
[...]
...The meanings of the words verification and validation have themselves been a problem (Oreskes et al., 1994)....
[...]
...The rational underpinnings of validation concepts are seldom considered, yet they are a strong influence on how we view the testing of models (e.g., Oreskes et al., 1994)....
[...]
Journal Article•
TL;DR: There is increasing evidence that demographic time and evolu tionary time are commensurate and population biology must deal simultaneously with genetic, physiological, and age heterogeneity within species of multispecies systems changing demographically and evolving under the fluctuating influences of other species in a heterogeneous environment.
Abstract: what were previously independent clusters of more or less co herent theory. Population genetics and population ecology, the most mathematical areas of population biology, had developed with quite different assumptions and techniques, while mathematical biogeography is essentially a new field. For population genetics, a population is specified by the frequencies of genotypes without reference to the age distribution, physiological state as a reflection of past history, or population density. A single population or species is treated at a time, and evolution is usually as sumed to occur in a constant environment. Population ecology, on the other hand, recognizes multispecies sys tems, describes populations in terms of their age distributions, phys iological states, and densities. The environment is allowed to vary but the species are treated as genetically homogeneous, so that evolution is ignored. But there is increasing evidence that demographic time and evolu tionary time are commensurate. Thus population biology must deal simultaneously with genetic, physiological, and age heterogeneity within species of multispecies systems changing demographically and evolving under the fluctuating influences of other species in a heterogeneous environment. The problem is how to deal with such a complex system. The naive, brute force approach would be to set up a mathematical model which is a faithful, one-to-one reflection of this complexity. This would require using perhaps 100 simultaneous partial differential equa tions with time lags; measuring hundreds of parameters, solving the equations to get numerical predictions, and then measuring these pre dictions against nature. However: (a) there are too many parameters to measure; some are still only vaguely defined; many would require a lifetime each for their measurement.
1,765 citations
"Testing ecological models: the mean..." refers background in this paper
...He expressed the view that models are hypotheses, which can only be falsified, and in this regard his opinion is opposite that of Levins (1966)....
[...]
...Levins (1966) initiated the discussion, “A mathematical model is neither an hypothesis nor a theory....
[...]