The multi-assumption architecture and testbed (MAAT v1.0): R code for generating ensembles with dynamic model structure and analysis of epistemic uncertainty from multiple sources
TL;DR: MAAT is a modular modelling code that can simply and efficiently vary model structure (process representation), allowing for the generation and running of large model ensembles that vary in process representation, parameters, parameter values, and environmental conditions during a single execution of the code.
Abstract: . Computer models are ubiquitous tools used to represent systems across many scientific and engineering domains. For any given system, many computer models exist, each built on different assumptions and demonstrating variability in the ways in which these systems can be represented. This variability is known as epistemic uncertainty, i.e. uncertainty in our knowledge of how these systems operate. Two primary sources of epistemic uncertainty are (1) uncertain parameter values and (2) uncertain mathematical representations of the processes that comprise the system. Many formal methods exist to analyse parameter-based epistemic uncertainty, while process-representation-based epistemic uncertainty is often analysed post hoc, incompletely, informally, or is ignored. In this model description paper we present the multi-assumption architecture and testbed (MAAT v1.0) designed to formally and completely analyse process-representation-based epistemic uncertainty. MAAT is a modular modelling code that can simply and efficiently vary model structure (process representation), allowing for the generation and running of large model ensembles that vary in process representation, parameters, parameter values, and environmental conditions during a single execution of the code. MAAT v1.0 approaches epistemic uncertainty through sensitivity analysis, assigning variability in model output to processes (process representation and parameters) or to individual parameters. In this model description paper we describe MAAT and, by using a simple groundwater model example, verify that the sensitivity analysis algorithms have been correctly implemented. The main system model currently coded in MAAT is a unified, leaf-scale enzyme kinetic model of C3 photosynthesis. In the Appendix we describe the photosynthesis model and the unification of multiple representations of photosynthetic processes. The numerical solution to leaf-scale photosynthesis is verified and examples of process variability in temperature response functions are provided. For rapid application to new systems, the MAAT algorithms for efficient variation of model structure and sensitivity analysis are agnostic of the specific system model employed. Therefore MAAT provides a tool for the development of novel or toy models in many domains, i.e. not only photosynthesis, facilitating rapid informal and formal comparison of alternative modelling approaches.
Citations
More filters
National Center for Atmospheric Research1, Lawrence Berkeley National Laboratory2, Oak Ridge National Laboratory3, Utrecht University4, Columbia University5, Chinese Academy of Sciences6, University of Houston7, California Institute of Technology8, University of California, Los Angeles9, Institute of Arctic and Alpine Research10, Los Alamos National Laboratory11, Harvard University12, Cooperative Institute for Research in Environmental Sciences13, University of Arizona14, University of Colorado Boulder15, Purdue University16, Michigan State University17, Argonne National Laboratory18, University of Michigan19, Auburn University20, Pacific Northwest National Laboratory21, Goddard Space Flight Center22, University of California, Irvine23, Virginia Tech24, University of Sheffield25
TL;DR: The Community Land Model (CLM) is the land component of the Community Earth System Model (CESM) and is used in several global and regional modeling systems.
Abstract: The Community Land Model (CLM) is the land component of the Community Earth System Model (CESM) and is used in several global and regional modeling systems. In this paper, we introduce model developments included in CLM version 5 (CLM5), which is the default land component for CESM2. We assess an ensemble of simulations, including prescribed and prognostic vegetation state, multiple forcing data sets, and CLM4, CLM4.5, and CLM5, against a range of metrics including from the International Land Model Benchmarking (ILAMBv2) package. CLM5 includes new and updated processes and parameterizations: (1) dynamic land units, (2) updated parameterizations and structure for hydrology and snow (spatially explicit soil depth, dry surface layer, revised groundwater scheme, revised canopy interception and canopy snow processes, updated fresh snow density, simple firn model, and Model for Scale Adaptive River Transport), (3) plant hydraulics and hydraulic redistribution, (4) revised nitrogen cycling (flexible leaf stoichiometry, leaf N optimization for photosynthesis, and carbon costs for plant nitrogen uptake), (5) global crop model with six crop types and time‐evolving irrigated areas and fertilization rates, (6) updated urban building energy, (7) carbon isotopes, and (8) updated stomatal physiology. New optional features include demographically structured dynamic vegetation model (Functionally Assembled Terrestrial Ecosystem Simulator), ozone damage to plants, and fire trace gas emissions coupling to the atmosphere. Conclusive establishment of improvement or degradation of individual variables or metrics is challenged by forcing uncertainty, parametric uncertainty, and model structural complexity, but the multivariate metrics presented here suggest a general broad improvement from CLM4 to CLM5.
661 citations
Cites methods from "The multi-assumption architecture a..."
...…and LAI reasonably well even with such low Vcmax25 values is potentially indicative of a structural problem in the leaf‐level versus canopy‐scaled value (as discussed in Rogers et al., 2017) which will be investigated further using off‐line tools such as those presented by Walker et al. (2018)....
[...]
01 Dec 2004
TL;DR: In this article, a framework is presented for assessing the predictive uncertainties of environmental models used for extrapolation, which involves the use of multiple conceptual models, assessment of their pedigree and reflection on the extent to which the sampled models adequately represent the space of plausible models.
Abstract: Although uncertainty about structures of environmental models (conceptual uncertainty) is often acknowledged to be the main source of uncertainty in model predictions, it is rarely considered in environmental modelling. Rather, formal uncertainty analyses have traditionally focused on model parameters and input data as the principal source of uncertainty in model predictions. The traditional approach to model uncertainty analysis, which considers only a single conceptual model, may fail to adequately sample the relevant space of plausible conceptual models. As such, it is prone to modelling bias and underestimation of predictive uncertainty. In this paper we review a range of strategies for assessing structural uncertainties in models. The existing strategies fall into two categories depending on whether field data are available for the predicted variable of interest. To date, most research has focussed on situations where inferences on the accuracy of a model structure can be made directly on the basis of field data. This corresponds to a situation of ‘interpolation’. However, in many cases environmental models are used for ‘extrapolation’; that is, beyond the situation and the field data available for calibration. In the present paper, a framework is presented for assessing the predictive uncertainties of environmental models used for extrapolation. It involves the use of multiple conceptual models, assessment of their pedigree and reflection on the extent to which the sampled models adequately represent the space of plausible models. � 2005 Elsevier Ltd. All rights reserved.
417 citations
Oak Ridge National Laboratory1, University of New South Wales2, Ludwig Maximilian University of Munich3, University of Arizona4, Stanford University5, Scripps Institution of Oceanography6, Smithsonian Environmental Research Center7, University of Sydney8, Max Planck Society9, Smithsonian Conservation Biology Institute10, Smithsonian Tropical Research Institute11, Seconda Università degli Studi di Napoli12, University of Leeds13, Aix-Marseille University14, Swiss Federal Institute for Forest, Snow and Landscape Research15, University of California, Santa Barbara16, Commonwealth Scientific and Industrial Research Organisation17, Université Paris-Saclay18, Australian National University19, National University of Singapore20, ETH Zurich21, California Institute of Technology22, Imperial College London23, Northern Arizona University24, Oeschger Centre for Climate Change Research25, Lawrence Berkeley National Laboratory26, University of California, Berkeley27, University of Basel28, Auckland University of Technology29, Indiana University30, University of Oxford31, Spanish National Research Council32, Umeå University33, University of Exeter34, Lawrence Livermore National Laboratory35, University of California, Irvine36, United States Geological Survey37, State University of New York College of Environmental Science and Forestry38, Rutgers University39, Wageningen University and Research Centre40
TL;DR: A range of evidence supports a positive terrestrial carbon sink in response to iCO2, albeit with uncertain magnitude and strong suggestion of a role for additional agents of global change.
Abstract: Atmospheric carbon dioxide concentration ([CO2 ]) is increasing, which increases leaf-scale photosynthesis and intrinsic water-use efficiency. These direct responses have the potential to increase plant growth, vegetation biomass, and soil organic matter; transferring carbon from the atmosphere into terrestrial ecosystems (a carbon sink). A substantial global terrestrial carbon sink would slow the rate of [CO2 ] increase and thus climate change. However, ecosystem CO2 responses are complex or confounded by concurrent changes in multiple agents of global change and evidence for a [CO2 ]-driven terrestrial carbon sink can appear contradictory. Here we synthesize theory and broad, multidisciplinary evidence for the effects of increasing [CO2 ] (iCO2 ) on the global terrestrial carbon sink. Evidence suggests a substantial increase in global photosynthesis since pre-industrial times. Established theory, supported by experiments, indicates that iCO2 is likely responsible for about half of the increase. Global carbon budgeting, atmospheric data, and forest inventories indicate a historical carbon sink, and these apparent iCO2 responses are high in comparison to experiments and predictions from theory. Plant mortality and soil carbon iCO2 responses are highly uncertain. In conclusion, a range of evidence supports a positive terrestrial carbon sink in response to iCO2 , albeit with uncertain magnitude and strong suggestion of a role for additional agents of global change.
234 citations
Cites background from "The multi-assumption architecture a..."
...Agile and extensible models (e.g. Clark et al., 2015; Walker et al., 2018) will be needed to rapidly incorporate this understanding, including uncertainty, into the internally-consistent and quantitative systems-level theory that models represent....
[...]
TL;DR: This review identifies three “grand challenges” in the development and use of LSMs, based around these issues: managing process complexity, representing land surface heterogeneity, and understanding parametric dynamics across the broad set of problems asked of LS Ms in a changing world.
Abstract: Author(s): Fisher, RA; Koven, CD | Abstract: Land surface models (LSMs) are a vital tool for understanding, projecting, and predicting the dynamics of the land surface and its role within the Earth system, under global change. Driven by the need to address a set of key questions, LSMs have grown in complexity from simplified representations of land surface biophysics to encompass a broad set of interrelated processes spanning the disciplines of biophysics, biogeochemistry, hydrology, ecosystem ecology, community ecology, human management, and societal impacts. This vast scope and complexity, while warranted by the problems LSMs are designed to solve, has led to enormous challenges in understanding and attributing differences between LSM predictions. Meanwhile, the wide range of spatial scales that govern land surface heterogeneity, and the broad spectrum of timescales in land surface dynamics, create challenges in tractably representing processes in LSMs. We identify three “grand challenges” in the development and use of LSMs, based around these issues: managing process complexity, representing land surface heterogeneity, and understanding parametric dynamics across the broad set of problems asked of LSMs in a changing world. In this review, we discuss progress that has been made, as well as promising directions forward, for each of these challenges.
198 citations
Cites background from "The multi-assumption architecture a..."
..., 2016) water and energy budget models, leaf photosynthesis models (Walker et al., 2018), offline models of forest structure (Farrior et al....
[...]
...…scope such as basin‐scale (Clark, Nijssen, et al., 2015) or site‐scale (Coon et al., 2016) water and energy budget models, leaf photosynthesis models (Walker et al., 2018), offline models of forest structure (Farrior et al., 2016) (Moore et al., 2018), and soil biogeochemical “testbed” models…...
[...]
...…is in deciding how to aggregate processes into higher‐level submodels: While it may be straightforward to define alternate hypotheses for, say, models of stomatal conductance or within‐leaf carbon assimilation (Walker et al., 2018), other sets of processes may not be as unambiguously delineated....
[...]
...A further difficulty is in deciding how to aggregate processes into higher‐level submodels: While it may be straightforward to define alternate hypotheses for, say, models of stomatal conductance or within‐leaf carbon assimilation (Walker et al., 2018), other sets of processes may not be as unambiguously delineated....
[...]
TL;DR: In this article, the authors propose a Bayesian model emulation of sufficient statistics, which can approximate the full joint posterior density, is more amenable to parallelization, and provides an estimate of parameter sensitivity.
Abstract: . Data-model integration plays a critical role in assessing and improving our capacity to predict ecosystem dynamics. Similarly, the ability to attach quantitative statements of uncertainty around model forecasts is crucial for model assessment and interpretation and for setting field research priorities. Bayesian methods provide a rigorous data assimilation framework for these applications, especially for problems with multiple data constraints. However, the Markov chain Monte Carlo (MCMC) techniques underlying most Bayesian calibration can be prohibitive for computationally demanding models and large datasets. We employ an alternative method, Bayesian model emulation of sufficient statistics, that can approximate the full joint posterior density, is more amenable to parallelization, and provides an estimate of parameter sensitivity. Analysis involved informative priors constructed from a meta-analysis of the primary literature and specification of both model and data uncertainties, and it introduced novel approaches to autocorrelation corrections on multiple data streams and emulating the sufficient statistics surface. We report the integration of this method within an ecological workflow management software, Predictive Ecosystem Analyzer (PEcAn), and its application and validation with two process-based terrestrial ecosystem models: SIPNET and ED2. In a test against a synthetic dataset, the emulator was able to retrieve the true parameter values. A comparison of the emulator approach to standard brute-force MCMC involving multiple data constraints showed that the emulator method was able to constrain the faster and simpler SIPNET model's parameters with comparable performance to the brute-force approach but reduced computation time by more than 2 orders of magnitude. The emulator was then applied to calibration of the ED2 model, whose complexity precludes standard (brute-force) Bayesian data assimilation techniques. Both models are constrained after assimilation of the observational data with the emulator method, reducing the uncertainty around their predictions. Performance metrics showed increased agreement between model predictions and data. Our study furthers efforts toward reducing model uncertainties, showing that the emulator method makes it possible to efficiently calibrate complex models.
68 citations
References
More filters
TL;DR: Various aspects of the biochemistry of photosynthetic carbon assimilation in C3 plants are integrated into a form compatible with studies of gas exchange in leaves.
Abstract: Various aspects of the biochemistry of photosynthetic carbon assimilation in C3 plants are integrated into a form compatible with studies of gas exchange in leaves. These aspects include the kinetic properties of ribulose bisphosphate carboxylase-oxygenase; the requirements of the photosynthetic carbon reduction and photorespiratory carbon oxidation cycles for reduced pyridine nucleotides; the dependence of electron transport on photon flux and the presence of a temperature dependent upper limit to electron transport. The measurements of gas exchange with which the model outputs may be compared include those of the temperature and partial pressure of CO2(p(CO2)) dependencies of quantum yield, the variation of compensation point with temperature and partial pressure of O2(p(O2)), the dependence of net CO2 assimilation rate on p(CO2) and irradiance, and the influence of p(CO2) and irradiance on the temperature dependence of assimilation rate.
7,312 citations
"The multi-assumption architecture a..." refers background or methods in this paper
...5(1− f ) using the Farquhar et al. (1980) notation....
[...]
...(A2b) & (A2c) Photorespiration rate at Tl Function of RuBisCO kinetic constants Farquhar et al. (1980) Eq....
[...]
...(A4) TPU-limited potential gross carbon assimilation rate Michaelis–Menten enzyme kinetics Farquhar et al. (1980) Eq....
[...]
...Enzyme kinetic models of leaf photosynthesis (Farquhar et al., 1980; Collatz et al., 1991; von Caemmerer, 2000) simulate net CO2 assimilation (A, µmolCO2 m−2 s−1) as the gross carboxylation rate (Ag, µmolCO2 m−2 s−1) scaled to account for the photorespiratory compensation point (0∗, Pa; the…...
[...]
...Process Assumption and/or hypothesis Citation RuBP-saturated potential gross carbon assimilation rate Michaelis–Menten enzyme kinetics Farquhar et al. (1980) Eq....
[...]
TL;DR: In this article, the authors propose a new framework for the construction of reversible Markov chain samplers that jump between parameter subspaces of differing dimensionality, which is flexible and entirely constructive.
Abstract: Markov chain Monte Carlo methods for Bayesian computation have until recently been restricted to problems where the joint distribution of all variables has a density with respect to some fixed standard underlying measure. They have therefore not been available for application to Bayesian model determination, where the dimensionality of the parameter vector is typically not fixed. This paper proposes a new framework for the construction of reversible Markov chain samplers that jump between parameter subspaces of differing dimensionality, which is flexible and entirely constructive. It should therefore have wide applicability in model determination problems. The methodology is illustrated with applications to multiple change-point analysis in one and two dimensions, and to a Bayesian comparison of binomial experiments.
6,188 citations
"The multi-assumption architecture a..." refers methods in this paper
...Markov chain Monte Carlo (MCMC) is a powerful Bayesian technique to estimate parameters and that can be used to select models, incorporating multiple sources of uncertainty (e.g. Vrugt et al., 2009; Green, 1995; Beven and Freer, 2001)....
[...]
...Markov chain Monte Carlo (MCMC) is a powerful Bayesian technique to estimate parameters and that can be used to select models, incorporating multiple sources of uncertainty (e.g. Vrugt et al., 2009; Green, 1995; Beven and Freer, 2001)....
[...]
University of Colorado Boulder1, Earth System Research Laboratory2, Met Office3, National Oceanic and Atmospheric Administration4, University of Bern5, ETH Zurich6, Rovira i Virgili University7, University of East Anglia8, University Corporation for Atmospheric Research9, South African Weather Service10, British Antarctic Survey11, University of Milan12, Norwegian Meteorological Institute13, University of Lisbon14, Environment Canada15, National Center for Atmospheric Research16
TL;DR: The Twentieth Century Reanalysis (20CR) dataset as discussed by the authors provides the first estimates of global tropospheric variability, and of the dataset's time-varying quality, from 1871 to the present at 6-hourly temporal and 2° spatial resolutions.
Abstract: The Twentieth Century Reanalysis (20CR) project is an international effort to produce a comprehensive global atmospheric circulation dataset spanning the twentieth century, assimilating only surface pressure reports and using observed monthly sea-surface temperature and sea-ice distributions as boundary conditions. It is chiefly motivated by a need to provide an observational dataset with quantified uncertainties for validations of climate model simulations of the twentieth century on all time-scales, with emphasis on the statistics of daily weather. It uses an Ensemble Kalman Filter data assimilation method with background ‘first guess’ fields supplied by an ensemble of forecasts from a global numerical weather prediction model. This directly yields a global analysis every 6 hours as the most likely state of the atmosphere, and also an uncertainty estimate of that analysis.
The 20CR dataset provides the first estimates of global tropospheric variability, and of the dataset's time-varying quality, from 1871 to the present at 6-hourly temporal and 2° spatial resolutions. Intercomparisons with independent radiosonde data indicate that the reanalyses are generally of high quality. The quality in the extratropical Northern Hemisphere throughout the century is similar to that of current three-day operational NWP forecasts. Intercomparisons over the second half-century of these surface-based reanalyses with other reanalyses that also make use of upper-air and satellite data are equally encouraging.
It is anticipated that the 20CR dataset will be a valuable resource to the climate research community for both model validations and diagnostic studies. Some surprising results are already evident. For instance, the long-term trends of indices representing the North Atlantic Oscillation, the tropical Pacific Walker Circulation, and the Pacific–North American pattern are weak or non-existent over the full period of record. The long-term trends of zonally averaged precipitation minus evaporation also differ in character from those in climate model simulations of the twentieth century. Copyright © 2011 Royal Meteorological Society and Crown Copyright.
3,043 citations
"The multi-assumption architecture a..." refers background in this paper
...…1993), identify the biophysical factors controlling biological activity (e.g. Walker et al., 2017a), interpolate sparse observations (e.g. Compo et al., 2011), project responses of the Earth system to anthropogenic activity (e.g. Friedlingstein et al., 2014), predict aerodynamic flow…...
[...]
TL;DR: Surviving in certain environments clearly does not require maximising photosynthetic capacity for a given leaf nitrogen content, as variation reflects different strategies of nitrogen partitioning, the electron transport capacity per unit of chlorophyll and the specific activity of RuBP carboxylase.
Abstract: The photosynthetic capacity of leaves is related to the nitrogen content primarily bacause the proteins of the Calvin cycle and thylakoids represent the majority of leaf nitrogen. To a first approximation, thylakoid nitrogen is proportional to the chlorophyll content (50 mol thylakoid N mol-1 Chl). Within species there are strong linear relationships between nitrogen and both RuBP carboxylase and chlorophyll. With increasing nitrogen per unit leaf area, the proportion of total leaf nitrogen in the thylakoids remains the same while the proportion in soluble protein increases. In many species, growth under lower irradiance greatly increases the partitioning of nitrogen into chlorophyll and the thylakoids, while the electron transport capacity per unit of chlorophyll declines. If growth irradiance influences the relationship between photosynthetic capacity and nitrogen content, predicting nitrogen distribution between leaves in a canopy becomes more complicated. When both photosynthetic capacity and leaf nitrogen content are expressed on the basis of leaf area, considerable variation in the photosynthetic capacity for a given leaf nitrogen content is found between species. The variation reflects different strategies of nitrogen partitioning, the electron transport capacity per unit of chlorophyll and the specific activity of RuBP carboxylase. Survival in certain environments clearly does not require maximising photosynthetic capacity for a given leaf nitrogen content. Species that flourish in the shade partition relatively more nitrogen into the thylakoids, although this is associated with lower photosynthetic capacity per unit of nitrogen.
2,973 citations
"The multi-assumption architecture a..." refers background in this paper
...The N content of RuBisCO in a leaf contributes a substantial proportion of total leaf N (Evans, 1989)....
[...]
TL;DR: Existing and new practices for sensitivity analysis of model output are compared and recommendations on which to use are offered to help practitioners choose which techniques to use.
2,265 citations
"The multi-assumption architecture a..." refers methods in this paper
...…2018 www.geosci-model-dev.net/11/3159/2018/ These methods are often based on Monte Carlo (MC) techniques that run large ensembles of model simulations that sample parameter space, boundary condition space, and initial condition space (Saltelli et al., 2010; Song et al., 2015; Dai and Ye, 2015)....
[...]
...The algorithms are described in detail in Saltelli et al. (2010) and Dai et al. (2017) so we do not go into great detail here....
[...]
...For the parameter sensitivity algorithm (Jansen, 1999; Saltelli et al., 2010), two parameter sample matrices are constructed, A and B, both with n rows and np columns, where n and np are the number of samples and the number of parameters in the sensitivity analysis....
[...]
...For global parameter sensitivity analysis the algorithm developed by Saltelli et al. (2010) is employed....
[...]
...The first-order, Si , and total sensitivity, STi , indices are calculated after Jansen (1999); see Table 2 (Saltelli et al., 2010)....
[...]