Watershed models are powerful tools for simulating the effect of watershed processes and management on soil and water resources. However, no comprehensive guidance is available to facilitate model evaluation in terms of the accuracy of simulated data compared to measured flow and constituent values. Thus, the objectives of this research were to: (1) determine recommended model evaluation techniques (statistical and graphical), (2) review reported ranges of values and corresponding performance ratings for the recommended statistics, and (3) establish guidelines for model evaluation based on the review results and project-specific considerations; all of these objectives focus on simulation of streamflow and transport of sediment and nutrients. These objectives were achieved with a thorough review of relevant literature on model application and recommended model evaluation methods. Based on this analysis, we recommend that three quantitative statistics, Nash-Sutcliffe efficiency (NSE), percent bias (PBIAS), and ratio of the root mean square error to the standard deviation of measured data (RSR), in addition to the graphical techniques, be used in model evaluation. The following model evaluation performance ratings were established for each recommended statistic. In general, model simulation can be judged as satisfactory if NSE > 0.50 and RSR < 0.70, and if PBIAS + 25% for streamflow, PBIAS + 55% for sediment, and PBIAS + 70% for N and P. For PBIAS, constituent-specific performance ratings were determined based on uncertainty of measured data. Additional considerations related to model evaluation guidelines are also discussed. These considerations include: single-event simulation, quality and quantity of measured data, model calibration procedure, evaluation time step, and project scope and magnitude. A case study illustrating the application of the model evaluation guidelines is also provided.

Model Evaluation Guidelines for Systematic Quantification of Accuracy in Watershed Simulations

This paper describes a methodology for calibration and uncertainty estimation of distributed models based on generalized likelihood measures. The GLUE procedure works with multiple sets of parameter values and allows that, within the limitations of a given model structure and errors in boundary conditions and field observations, different sets of values may be equally likely as simulators of a catchment. Procedures for incorporating different types of observations into the calibration; Bayesian updating of likelihood values and evaluating the value of additional observations to the calibration process are described. The procedure is computationally intensive but has been implemented on a local parallel processing computer.

The future of distributed models: model calibration and uncertainty prediction.

Decomposition of the mean squared error and NSE performance criteria: Implications for improving hydrological modelling

The successful application of a conceptual rainfall-runoff (CRR) model depends on how well it is calibrated. Despite the popularity of CRR models, reports in the literature indicate that it is typically difficult, if not impossible, to obtain unique optimal values for their parameters using automatic calibration methods. Unless the best set of parameters associated with a given calibration data set can be found, it is difficult to determine how sensitive the parameter estimates (and hence the model forecasts) are to factors such as input and output data error, model error, quantity and quality of data, objective function used, and so on. Results are presented that establish clearly the nature of the multiple optima problem for the research CRR model SIXPAR. These results suggest that the CRR model optimization problem is more difficult than had been previously thought and that currently used local search procedures have a very low probability of successfully finding the optimal parameter sets. Next, the performance of three existing global search procedures are evaluated on the model SIXPAR. Finally, a powerful new global optimization procedure is presented, entitled the shuffled complex evolution (SCE-UA) method, which was able to consistently locate the global optimum of the SIXPAR model, and appears to be capable of efficiently and effectively solving the CRR model optimization problem.

Effective and efficient global optimization for conceptual rainfall‐runoff models

SWAT (Soil and Water Assessment Tool) is a comprehensive, semi-distributed river basin model that requires a large number of input parameters, which complicates model parameterization and calibration. Several calibration techniques have been developed for SWAT, including manual calibration procedures and automated procedures using the shuffled complex evolution method and other common methods. In addition, SWAT-CUP was recently developed and provides a decision-making framework that incorporates a semi-automated approach (SUFI2) using both manual and automated calibration and incorporating sensitivity and uncertainty analysis. In SWAT-CUP, users can manually adjust parameters and ranges iteratively between autocalibration runs. Parameter sensitivity analysis helps focus the calibration and uncertainty analysis and is used to provide statistics for goodness-of-fit. The user interaction or manual component of the SWAT-CUP calibration forces the user to obtain a better understanding of the overall hydrologic processes (e.g., baseflow ratios, ET, sediment sources and sinks, crop yields, and nutrient balances) and of parameter sensitivity. It is important for future calibration developments to spatially account for hydrologic processes; improve model run time efficiency; include the impact of uncertainty in the conceptual model, model parameters, and measured variables used in calibration; and assist users in checking for model errors. When calibrating a physically based model like SWAT, it is important to remember that all model input parameters must be kept within a realistic uncertainty range and that no automatic procedure can substitute for actual physical knowledge of the watershed.

/pdf/swat-model-use-calibration-and-validation-3yiyw8b1pn.pdf

SWAT: Model Use, Calibration, and Validation

The degree of difficulty in solving a global optimization problem is in general dependent on the dimensionality of the problem and certain characteristics of the objective function. This paper discusses five of these characteristics and presents a strategy for function optimization called the shuffled complex evolution (SCE) method, which promises to be robust, effective, and efficient for a broad class of problems. The SCE method is based on a synthesis of four concepts that have proved successful for global optimization: (a) combination of probabilistic and deterministic approaches; (b) clustering; (c) systematic evolution of a complex of points spanning the space, in the direction of global improvement; and (d) competitive evolution. Two algorithms based on the SCE method are presented. These algorithms are tested by running 100 randomly initiated trials on eight test problems of differing difficulty. The performance of the two algorithms is compared to that of the controlled random search CRS2 method presented by Price (1983, 1987) and to a multistart algorithm based on the simplex method presented by Nelder and Mead (1965).

Shuffled complex evolution approach for effective and efficient global minimization

https://swat.tamu.edu/media/90111/sdarticle.pdf

Optimal use of the SCE-UA global optimization method for calibrating watershed models

Conceptual rainfall-runoff models are difficult to calibrate by means of automatic methods; one major reason for this is the inability of conventional procedures to locate the globally optimal set of parameters. This paper investigates the consistency with which two global optimization methods, the shuffled complex evolution (SCE-UA) method (developed by the authors) and the multistart simplex (MSX) method, are able to find the optimal parameter set during calibration of the Sacramento soil moisture accounting model (SAC-SMA) of the National Weather Service River Forecast System (NWSRFS). In the first phase of this study, error-free synthetic data are used to conduct a comparative evaluation of the algorithms under “ideal” conditions. In 10 independent trials of each algorithm in which 13 parameters of the SAC-SMA model were optimized simultaneously, the SCE-UA method achieved a 100% success rate in locating the precise global optimum (i.e., the “true” parameter values) while the MSX method failed in all trials even with more than twice the number of function evaluations. In the second phase, historical data from the Leaf River watershed are used to conduct a comparative evaluation of the algorithms under “real” conditions, using two different estimation criteria, DRMS and HMLE; the SCE-UA algorithm obtained consistently lower function values and more closely grouped parameter estimates, while using one-third fewer function evaluations than the MSX algorithm.

Calibration of rainfall‐runoff models: Application of global optimization to the Sacramento Soil Moisture Accounting Model

The success of an automatic calibration procedure is highly dependent on the choice of the objective function and the nature (quantity and quality) of the data used. The objective function should be selected on the basis of the stochastic properties of the errors present in the data and in the model. Also, the data should be chosen so as to contain as much valuable information about the process as possible. In this paper we compare the performance of two maximum likelihood estimators, the AMLE, which assumes the presence of first lag autocorrelated homogeneous variance errors, and the HMLE, which assumes the presence of uncorrelated inhomogeneous variance errors, to the commonly used simple least squares criterion, SLS. The model calibrated was the soil moisture accounting model of the U.S. National Weather Service's river forecast system (SMA-NWSRFS). The results indicate that a properly chosen objective function can enhance the possibility of obtaining unique and conceptually realistic parameter estimates. Furthermore, the sensitivity of the estimation results to various characteristics of the calibration data, such as hydrologic variability and length, are substantially reduced.

Evaluation of Maximum Likelihood Parameter estimation techniques for conceptual rainfall‐runoff models: Influence of calibration data variability and length on model credibility

Vijai Kumar Gupta

Papers

Effective and efficient global optimization for conceptual rainfall‐runoff models

Shuffled complex evolution approach for effective and efficient global minimization

Optimal use of the SCE-UA global optimization method for calibrating watershed models

Calibration of rainfall‐runoff models: Application of global optimization to the Sacramento Soil Moisture Accounting Model

Evaluation of Maximum Likelihood Parameter estimation techniques for conceptual rainfall‐runoff models: Influence of calibration data variability and length on model credibility