The ODD protocol: A review and first update
TL;DR: The definition of ODD is revised to clarify aspects of the original version and thereby facilitate future standardization of ABM descriptions and improves the rigorous formulation of models and helps make the theoretical foundations of large models more visible.
About: This article is published in Ecological Modelling.The article was published on 2010-11-24 and is currently open access. It has received 2186 citations till now.
Citations
More filters
TL;DR: General classes of direct value comparison, coupling real and modelled values, preserving data patterns, indirect metrics based on parameter values, and data transformations are discussed.
Abstract: In order to use environmental models effectively for management and decision-making, it is vital to establish an appropriate level of confidence in their performance. This paper reviews techniques available across various fields for characterising the performance of environmental models with focus on numerical, graphical and qualitative methods. General classes of direct value comparison, coupling real and modelled values, preserving data patterns, indirect metrics based on parameter values, and data transformations are discussed. In practice environmental modelling requires the use and implementation of workflows that combine several methods, tailored to the model purpose and dependent upon the data and information available. A five-step procedure for performance evaluation of models is suggested, with the key elements including: (i) (re)assessment of the model's aim, scale and scope; (ii) characterisation of the data for calibration and testing; (iii) visual and other analysis to detect under- or non-modelled behaviour and to gain an overview of overall performance; (iv) selection of basic performance criteria; and (v) consideration of more advanced methods to handle problems such as systematic divergence between modelled and observed values.
1,207 citations
Cites background from "The ODD protocol: A review and firs..."
...In order to keep track of model changes, reflect on its context and communicate its purpose to new users, it may help to document the model description in a standardised way, such as that described by Grimm et al. (2006, 2010)....
[...]
TL;DR: The ODD + D protocol may prove helpful for describing ABMs in general when human decisions are included and incorporates a section on 'Theoretical and Empirical Background' to encourage model designs and model assumptions that are more closely related to theory.
Abstract: Representing human decisions is of fundamental importance in agent-based models. However, the rationale for choosing a particular human decision model is often not sufficiently empirically or theoretically substantiated in the model documentation. Furthermore, it is difficult to compare models because the model descriptions are often incomplete, not transparent and difficult to understand. Therefore, we expand and refine the 'ODD' (Overview, Design Concepts and Details) protocol to establish a standard for describing ABMs that includes human decision-making (ODD + D). Because the ODD protocol originates mainly from an ecological perspective, some adaptations are necessary to better capture human decision-making. We extended and rearranged the design concepts and related guiding questions to differentiate and describe decision-making, adaptation and learning of the agents in a comprehensive and clearly structured way. The ODD + D protocol also incorporates a section on 'Theoretical and Empirical Background' to encourage model designs and model assumptions that are more closely related to theory. The application of the ODD + D protocol is illustrated with a description of a social-ecological ABM on water use. Although the ODD + D protocol was developed on the basis of example implementations within the socio-ecological scientific community, we believe that the ODD + D protocol may prove helpful for describing ABMs in general when human decisions are included.
386 citations
Cites background or methods from "The ODD protocol: A review and firs..."
...It is more difficult to standardise this block across different disciplines and Grimm et al. (2010) already anticipated that the list of design conceptsmay need to be enlarged....
[...]
...In the following, we summarise the original description of the ODD protocol (see Grimm et al., 2010) and our extensions....
[...]
...Beyond the requirements formulated in the ODD protocol (Grimm et al., 2006, 2010), we strongly encourage that all questions be answered to avoid an incomplete model description....
[...]
...The first revision of the ODD protocol has attempted to open the standard for all ABMs (Grimm et al., 2010)....
[...]
...Expected benefits from ODD þ D Using standardised protocols to describe simulation models offers many advantages (see also Grimm et al., 2010): The experienced scientific audience can understand the models described with a standardised protocol more easily, and metaanalyses on existing models is…...
[...]
Book•
22 Jan 2013
TL;DR: In this paper, the authors present an overview of the practice of modelling and its application in the field of natural language processing, focusing on three types of models: concrete models, mathematical models, and computational models.
Abstract: Contents Preface 1 Introduction 1.1 Two Aquatic Puzzles 1.2 Models of Modeling 2 Three Kinds of Models 2.1 Concrete Model: The San Francisco Bay-Delta Model 2.2 Mathematical Model: Lotka-Volterra Model 2.3 Computational Model: Schelling's Segregation Model 2.4 Common Features of these Models 2.5 Only Three Types of Models? 2.6 Fewer Than Three Types of Model? 3 The Anatomy of Models: Structure & Construal 3.1 Structure 3.1.1 Concrete Structures 3.1.2 Mathematical 3.1.3 Computational 3.2 Model Descriptions 3.3 Construal 3.4 Representational Capacity of Structures 4 Fictions and Folk Ontology 4.1 Against Maths: Individuation, Causes, and Face Value Practice 4.2 A Simple Fictions Account 4.3 Enriching the Simple Account 4.3.1 Waltonian Fictionalism 4.3.2 Fictions without Models 4.4 Why I am not a Fictionalist 4.4.1 Variation 4.4.2 Representational Capacity of Different Models 4.4.3 Making Sense of Modeling 4.4.4 Variation in Practice 4.5 Folk ontology 4.6 Maths, Interpretation, and Folk Ontology 5 Target Directed Modeling 5.1 Model Development 5.2 Analysis of the Model 5.2.1 Complete Analysis 5.2.2 Goal-directed Analysis 5.3 Model/Target Comparison 5.3.1 Phenomena and Target Systems 5.3.2 Establishing the fit between Model and Target 5.3.3 Representations of Targets 6 Idealization 6.1 Three Kinds of Idealization 6.1.1 Galilean idealization 6.1.2 Minimalist idealization 6.1.3 Multiple Models Idealization 6.2 Representational Ideals and Fidelity Criteria 6.2.1 Completeness 6.2.2 Simplicity 6.2.3 1-Causal 6.2.4 Maxout 6.2.5 P-General 6.3 Idealization and Representational Ideals 6.4 Idealization and Target Directed Modeling 7 Modeling Without a Specific Target 7.1 Generalized Modeling 7.1.1 How Possibly Explanations 7.1.2 Minimal Models and First Order Causal Structures 7.2 Hypothetical Modeling 7.2.1 Contingent Non-existence: xDNA 7.2.2 Impossible Targets: Infinite Population Growth and Perpetual Motion 7.3 Targetless Modeling 7.4 A Moving Target: The Case of Three-sex Biology 8 An Account of Similarity 8.1 Desiderata for Model/World Relations 8.2 Model Theoretic Accounts 8.3 Similarity 8.4 Tversky's Contrast Account 8.5 Attributes and Mechanisms 8.6 Feature Sets, Construals, and Target Systems 8.7 Modeling Goals and Weighting Parameters 8.8 Weighting Function and Background Theory 8.9 Satisfying the Desiderata 9 Robustness Analysis and Idealization 9.1 Levins and Wimsatt on Robustness 9.2 Finding Robust Theorems 9.3 Three Kinds of Robustness 9.3.1 Parameter Robustness 9.3.2 Structural Robustness 9.3.3 Representational Robustness 9.4 Robustness and Confirmation 10 Conclusion: The Practice of Modeling References
379 citations
TL;DR: A mini-review of applications of POM confirms that making the selection and use of patterns more explicit and rigorous can facilitate the development of models with the right level of complexity to understand ecological systems and predict their response to novel conditions.
Abstract: Modern ecology recognizes that modelling systems across scales and at multiple levels—especially to link population and ecosystem dynamics to individual adaptive behaviour—is essential for making the science predictive. ‘Pattern-oriented modelling’ (POM) is a strategy for doing just this. POM is the multi-criteria design, selection and calibration of models of complex systems. POM starts with identifying a set of patterns observed at multiple scales and levels that characterize a system with respect to the particular problem being modelled; a model from which the patterns emerge should contain the right mechanisms to address the problem. These patterns are then used to (i) determine what scales, entities, variables and processes the model needs, (ii) test and select submodels to represent key low-level processes such as adaptive behaviour, and (iii) find useful parameter values during calibration. Patterns are already often used in these ways, but a mini-review of applications of POM confirms that making the selection and use of patterns more explicit and rigorous can facilitate the development of models with the right level of complexity to understand ecological systems and predict their response to novel conditions.
345 citations
TL;DR: This article presents a literature review of the methodologies applied in 60 dynamic MFAs of metals and focuses on giving a comprehensive overview of modeling approaches and structure them according to essential aspects, such as their treatment of material dissipation, spatial dimension of flows, or data uncertainty.
Abstract: Dynamic material flow analysis (MFA) is a frequently used method to assess past, present, and future stocks and flows of metals in the anthroposphere. Over the past fifteen years, dynamic MFA has contributed to increased knowledge about the quantities, qualities, and locations of metal-containing goods. This article presents a literature review of the methodologies applied in 60 dynamic MFAs of metals. The review is based on a standardized model description format, the ODD (overview, design concepts, details) protocol. We focus on giving a comprehensive overview of modeling approaches and structure them according to essential aspects, such as their treatment of material dissipation, spatial dimension of flows, or data uncertainty. The reviewed literature features similar basic modeling principles but very diverse extrapolation methods. Basic principles include the calculation of outflows of the in-use stock based on inflow or stock data and a lifetime distribution function. For extrapolating stocks and fl...
332 citations
References
More filters
TL;DR: A graphical method is discussed which allows a specification of the optimal diet of a predator in terms of the net amount of energy gained from a capture of prey as compared to the energy expended in searching for the prey.
Abstract: A graphical method is discussed which allows a specification of the optimal diet of a predator in terms of the net amount of energy gained from a capture of prey as compared to the energy expended in searching for the prey. The method allows several predictions about changes in the degree of specialization of the diet as the numbers of different prey organisms change. For example, a more productive environment should lead to more restricted diet in numbers of different species eaten. In a patchy environment, however, this will not apply to predators that spend most of their time searching. Moreover, larger patches are used in a more specialized way than smaller patches.
4,132 citations
3,158 citations
TL;DR: A proposed standard protocol for describing IBMs and ABMs, developed and tested by 28 modellers who cover a wide range of fields within ecology, and considered as a first step for establishing a more detailed common format of the description of IBm and ABM.
2,633 citations
"The ODD protocol: A review and firs..." refers background or methods in this paper
...These criticisms motivated the ODD (Overview, Design concepts, Details) protocol (Grimm et al., 2006), which attempted to create a generic format and a standard structure by which all ABMs could be documented....
[...]
...We searched the ‘Web of Science’ reference data base (Thomson Scientific) for publications citing the original ODD publication (Grimm et al., 2006)....
[...]
...…publications with ODD model descriptions Supplemental material for Exercise 1, Chapter 3 of Agent-Based and Individual-Based Modeling More examples can be found by searching for publications that cite the ODD papers (although not all papers that cite these actually use ODD): Grimm et al. 2006....
[...]
...Elements of the original ODD protocol (Grimm et al., 2006) Elements of the updated ODD protocol...
[...]
TL;DR: This paper argues that recent advances in ecological modeling have come together in a general strategy that provides a unifying framework for decoding the internal organization of agent-based complex systems and may lead toward unifying algorithmic theories of the relation between adaptive behavior and system complexity.
Abstract: Agent-based complex systems are dynamic networks of many interacting agents; examples include ecosystems, financial markets, and cities. The search for general principles underlying the internal organization of such systems often uses bottom-up simulation models such as cellular automata and agent-based models. No general framework for designing, testing, and analyzing bottom-up models has yet been established, but recent advances in ecological modeling have come together in a general strategy we call pattern-oriented modeling. This strategy provides a unifying framework for decoding the internal organization of agent-based complex systems and may lead toward unifying algorithmic theories of the relation between adaptive behavior and system complexity.
1,933 citations
Book•
01 Jan 2005
TL;DR: An excellent introduction and overview of this field, written by Volker Grimm and Steven F. Railsback, should be read by everyone interested in individual-based modeling and especially by anyone contemplating developing, or being involved with a group developing, an individualbased model.
Abstract: Individual-based modeling is a new, exciting discipline that allows ecologists to explore, using computer simulations, how properties of populations and ecosystems might evolve from the characteristics and behaviors of individual organisms. Individual-based Modeling and Ecology, written by Volker Grimm and Steven F. Railsback, gives an excellent introduction and overview of this field. It should be read by everyone interested in individual-based modeling, and especially by anyone contemplating developing, or being involved with a group developing, an individualbased model.
1,495 citations
"The ODD protocol: A review and firs..." refers background in this paper
...Therefore, they are included in ODD as a kind of checklist to make sure that important model design decisions are made consciously and that readers are aware of these decisions (Railsback, 2001; Grimm and Railsback, 2005)....
[...]
...) ABMs focus on one or more of the following aspects because they are considered critical for explaining system-level behavior: heterogeneity of and among individuals, local interactions among individuals, and adaptive behavior of individuals (DeAngelis and Mooij, 2003, 2005; Grimm and Railsback, 2005)....
[...]
...(Decisions made throughout the modeling cycle [Grimm and Railsback, 2005] are likely to be iterative....
[...]