Author
Cort J. Willmott
Bio: Cort J. Willmott is an academic researcher from University of Delaware. The author has contributed to research in topics: Precipitation & Spatial variability. The author has an hindex of 31, co-authored 60 publications receiving 17784 citations.
Papers published on a yearly basis
Papers
More filters
TL;DR: In this paper, it is suggested that the correlation coefficieness between observed and simulated variates is not as good as observed variates, and that correlation can be improved.
Abstract: Traditional methods of evaluating geographic models by statistical comparisons between observed and simulated variates are criticized. In particular, it is suggested that the correlation coefficien...
3,761 citations
TL;DR: In this paper, the root-mean-square error (RMSE) and the mean absolute error (MAE) were examined to describe average model-performance error, and it was shown that MAE is a more natural measure of average error than RMSE.
Abstract: The relative abilities of 2, dimensioned statistics — the root-mean-square error (RMSE) and the mean absolute error (MAE) — to describe average model-performance error are examined The RMSE is of special interest because it is widely reported in the climatic and environmental liter- ature; nevertheless, it is an inappropriate and misinterpreted measure of average error RMSE is inappropriate because it is a function of 3 characteristics of a set of errors, rather than of one (the average error) RMSE varies with the variability within the distribution of error magnitudes and with the square root of the number of errors (n 1/2 ), as well as with the average-error magnitude (MAE) Our findings indicate that MAE is a more natural measure of average error, and (unlike RMSE) is unambiguous Dimensioned evaluations and inter-comparisons of average model-performance error, therefore, should be based on MAE
3,500 citations
TL;DR: In this article, it is suggested that the correlation between model-predicted and observed data, commonly described by Pearson's productmoment correlation coefficient, is an insufficient and often misleading measure of accuracy, and a complement of difference and summary univariate indices is presented as the nucleus of a more informative, albeit fundamentally descriptive, approach to model evaluation.
Abstract: Quantitative approaches to the evaluation of model performance were recently examined by Fox (1981). His recommendations are briefly reviewed and a revised set of performance statistics is proposed. It is suggested that the correlation between model-predicted and observed data, commonly described by Pearson's product-moment correlation coefficient, is an insufficient and often misleading measure of accuracy. A complement of difference and summary univariate indices is presented as the nucleus of a more informative, albeit fundamentally descriptive, approach to model evaluation. Two models that estimate monthly evapotranspiration are comparatively evaluated in order to illustrate how the recommended method(s) can be applied.
3,218 citations
TL;DR: In this paper, a set of difference measures are used to evaluate the operational performance of a wide spectrum of geophysical models, regardless of whether the model predictions are manifested as scalars, directions, or vectors.
Abstract: Procedures that may be used to evaluate the operational performance of a wide spectrum of geophysical models are introduced. Primarily using a complementary set of difference measures, both model accuracy and precision can be meaningfully estimated, regardless of whether the model predictions are manifested as scalars, directions, or vectors. It is additionally suggested that the reliability of the accuracy and precision measures can be determined from bootstrap estimates of confidence and significance. Recommended procedures are illustrated with a comparative evaluation of two models that estimate wind velocity over the South Atlantic Bight.
1,832 citations
TL;DR: In this article, a global climatology of mean monthly precipitation has been developed using traditional land-based gauge measurements and shipboard estimates, and a procedure for correcting gauge-induced biases is presented and used to remove systematic errors caused by wind, wetting on the interior walls of the gauge, and evaporation from the gauge.
Abstract: Using traditional land-based gauge measurements and shipboard estimates, a global climatology of mean monthly precipitation has been developed. Data were obtained from ten existing sources, screened for coding errors, and redundant station records were removed. The edited data base contains 24,635 spatially independent terrestrial station records and 2223 oceanic grid-point records. A procedure for correcting gauge-induced biases is presented and used to remove systematic errors caused by wind, wetting on the interior walls of the gauge, and evaporation from the gauge. These ‘corrected’ monthly precipitation observations were then interpolated to a 0·5° of latitude by 0·5° of longitude grid using a spherically based interpolation procedure. Bias-corrected spatial distributions of the annual mean and intraannual variance are presented along with a harmonic decomposition of the intra-annual variance.
1,551 citations
Cited by
More filters
TL;DR: The NCEP/NCAR 40-yr reanalysis uses a frozen state-of-the-art global data assimilation system and a database as complete as possible, except that the horizontal resolution is T62 (about 210 km) as discussed by the authors.
Abstract: The NCEP and NCAR are cooperating in a project (denoted “reanalysis”) to produce a 40-year record of global analyses of atmospheric fields in support of the needs of the research and climate monitoring communities. This effort involves the recovery of land surface, ship, rawinsonde, pibal, aircraft, satellite, and other data; quality controlling and assimilating these data with a data assimilation system that is kept unchanged over the reanalysis period 1957–96. This eliminates perceived climate jumps associated with changes in the data assimilation system. The NCEP/NCAR 40-yr reanalysis uses a frozen state-of-the-art global data assimilation system and a database as complete as possible. The data assimilation and the model used are identical to the global system implemented operationally at the NCEP on 11 January 1995, except that the horizontal resolution is T62 (about 210 km). The database has been enhanced with many sources of observations not available in real time for operations, provided b...
28,145 citations
TL;DR: In this paper, the authors present guidelines for watershed model evaluation based on the review results and project-specific considerations, including single-event simulation, quality and quantity of measured data, model calibration procedure, evaluation time step, and project scope and magnitude.
Abstract: Watershed models are powerful tools for simulating the effect of watershed processes and management on soil and water resources. However, no comprehensive guidance is available to facilitate model evaluation in terms of the accuracy of simulated data compared to measured flow and constituent values. Thus, the objectives of this research were to: (1) determine recommended model evaluation techniques (statistical and graphical), (2) review reported ranges of values and corresponding performance ratings for the recommended statistics, and (3) establish guidelines for model evaluation based on the review results and project-specific considerations; all of these objectives focus on simulation of streamflow and transport of sediment and nutrients. These objectives were achieved with a thorough review of relevant literature on model application and recommended model evaluation methods. Based on this analysis, we recommend that three quantitative statistics, Nash-Sutcliffe efficiency (NSE), percent bias (PBIAS), and ratio of the root mean square error to the standard deviation of measured data (RSR), in addition to the graphical techniques, be used in model evaluation. The following model evaluation performance ratings were established for each recommended statistic. In general, model simulation can be judged as satisfactory if NSE > 0.50 and RSR < 0.70, and if PBIAS + 25% for streamflow, PBIAS + 55% for sediment, and PBIAS + 70% for N and P. For PBIAS, constituent-specific performance ratings were determined based on uncertainty of measured data. Additional considerations related to model evaluation guidelines are also discussed. These considerations include: single-event simulation, quality and quantity of measured data, model calibration procedure, evaluation time step, and project scope and magnitude. A case study illustrating the application of the model evaluation guidelines is also provided.
9,386 citations
TL;DR: A review of predictive habitat distribution modeling is presented, which shows that a wide array of models has been developed to cover aspects as diverse as biogeography, conservation biology, climate change research, and habitat or species management.
6,748 citations
TL;DR: In this paper, an updated gridded climate dataset (referred to as CRU TS3.10) from monthly observations at meteorological stations across the world's land areas is presented.
Abstract: This paper describes the construction of an updated gridded climate dataset (referred to as CRU TS3.10) from monthly observations at meteorological stations across the world's land areas. Station anomalies (from 1961 to 1990 means) were interpolated into 0.5° latitude/longitude grid cells covering the global land surface (excluding Antarctica), and combined with an existing climatology to obtain absolute monthly values. The dataset includes six mostly independent climate variables (mean temperature, diurnal temperature range, precipitation, wet-day frequency, vapour pressure and cloud cover). Maximum and minimum temperatures have been arithmetically derived from these. Secondary variables (frost day frequency and potential evapotranspiration) have been estimated from the six primary variables using well-known formulae. Time series for hemispheric averages and 20 large sub-continental scale regions were calculated (for mean, maximum and minimum temperature and precipitation totals) and compared to a number of similar gridded products. The new dataset compares very favourably, with the major deviations mostly in regions and/or time periods with sparser observational data. CRU TS3.10 includes diagnostics associated with each interpolated value that indicates the number of stations used in the interpolation, allowing determination of the reliability of values in an objective way. This gridded product will be publicly available, including the input station series (http://www.cru.uea.ac.uk/ and http://badc.nerc.ac.uk/data/cru/). © 2013 Royal Meteorological Society
5,552 citations
TL;DR: The Global Precipitation Climatology Project (GPCP) version 2 Monthly Precise Analysis as discussed by the authors is a merged analysis that incorporates precipitation estimates from low-orbit satellite microwave data, geosynchronous-orbit-satellite infrared data, and rain gauge observations.
Abstract: The Global Precipitation Climatology Project (GPCP) Version 2 Monthly Precipitation Analysis is described. This globally complete, monthly analysis of surface precipitation at 2.5 degrees x 2.5 degrees latitude-longitude resolution is available from January 1979 to the present. It is a merged analysis that incorporates precipitation estimates from low-orbit-satellite microwave data, geosynchronous-orbit-satellite infrared data, and rain gauge observations. The merging approach utilizes the higher accuracy of the low-orbit microwave observations to calibrate, or adjust, the more frequent geosynchronous infrared observations. The data set is extended back into the premicrowave era (before 1987) by using infrared-only observations calibrated to the microwave-based analysis of the later years. The combined satellite-based product is adjusted by the raingauge analysis. This monthly analysis is the foundation for the GPCP suite of products including those at finer temporal resolution, satellite estimate, and error estimates for each field. The 23-year GPCP climatology is characterized, along with time and space variations of precipitation.
4,951 citations