scispace - formally typeset
Search or ask a question

Showing papers on "Business Process Model and Notation published in 2008"


Journal ArticleDOI
TL;DR: In this article, the authors propose a mapping from BPMN to Petri nets, for which efficient analysis techniques are available, and implement a tool that, in conjunction with existing Petri net-based tools, enables the static analysis of BPMNs models.
Abstract: The Business Process Modelling Notation (BPMN) is a standard for capturing business processes in the early phases of systems development. The mix of constructs found in BPMN makes it possible to create models with semantic errors. Such errors are especially serious, because errors in the early phases of systems development are among the most costly and hardest to correct. The ability to statically check the semantic correctness of models is thus a desirable feature for modelling tools based on BPMN. Accordingly, this paper proposes a mapping from BPMN to a formal language, namely Petri nets, for which efficient analysis techniques are available. The proposed mapping has been implemented as a tool that, in conjunction with existing Petri net-based tools, enables the static analysis of BPMN models. The formalisation also led to the identification of deficiencies in the BPMN standard specification.

654 citations


Journal Article
TL;DR: This paper proposes a mapping from BPMN to a formal language, namely Petri nets, for which efficient analysis techniques are available and has been implemented as a tool that, in conjunction with existing Petri net-based tools, enables the static analysis of BPMn models.
Abstract: The Business Process Modelling Notation (BPMN) is a standard for capturing business processes in the early phases of system development. The mix of constructs in BPMN makes it possible to define models with a range of semantic errors. But the static analysis of BPMN models to detect such errors is hindered by ambiguities in the standard specification and the complexity of the language. The fact that BPMN integrates constructs from graph-oriented process definition languages with features for concurrent execution of multiple instances of a subprocess and exception handling, makes it challenging to provide a formal semantics of BPMN. Even more challenging is to provide a semantics that can be used to analyse BPMN models. This paper proposes a formalisation of BPMN in terms a mapping to Petri nets, for which efficient analysis techniques exist. The mapping has been implemented as a tool that generates code in the Petri Net Markup Language. The formalisation has led to the identification of deficiencies in the BPMN specification.

540 citations


Journal ArticleDOI
TL;DR: The Business Process Modeling Notation (BPMN) as discussed by the authors is a standard for graph-based representation of business process models, which is used in the BPMN 2.0 standard.
Abstract: Die Business Process Modeling Notation (BPMN) liefert einen Standard zur graphischen Beschreibung von Geschäftsprozessen. Der Standard wird von der Object Management Group (OMG) gepf legt und hat insbesondere im internationalen Raum eine erhebliche Aufmerksamkeit erlangt (Finkelstein 2005, S. 14). Der Namensbestandteil „Notation“ ist insofern irreführend, als dass die BPMN nicht nur ein System von Zeichen und Symbolen zur visuellen Repräsentation von Geschäftsprozessen beinhaltet, sondern auch Vorgaben zur Bedeutung und zur Verwendung der Zeichen und Symbole unterbreitet. Folglich ist die BPMN als eine Modellierungssprache zu verstehen. Hauptmotivation für die Entwicklung der BPMN war es, die unterschiedlichen Sprachen zur Geschäftsprozessmodellierung zu vereinheitlichen und so eine einzige, standardisierte Sprache zur Beschreibung von Geschäftsprozessen zur Verfügung zu stellen. Bei der Entwicklung der BPMN wurden unter anderem Aktivitätsdiagramme der Unified Modeling Lan­ guage (UML), IDEF, ebXML BPSS, RosettaNet und ereignisgesteuerte Prozessketten (EPK) explizit berücksichtigt (White 2004, S. 9). Darüber hinaus wollten die Entwickler der BPMN eine Prozessbeschreibungssprache schaffen, die es nicht nur ermöglicht, Prozessmodelle für den Fachanwender zu beschreiben, sondern die auch eine automatisierte Transformation fachlicher in technische Prozessmodelle unterstützt. Auf diesem Weg können Entwicklungsfehler vermieden und die Anwendungsentwicklung beschleunigt werden. Die BPMN wurde seit Anfang der 2000er-Jahre von der Business Process Management Initiative Notation Working Group (BPMI) unter der Führung von St. A. White, einem IBM-Mitarbeiter, entwickelt und 2004 erstmalig in der Version 1.0 vorgestellt. Im Jahr 2005 wurde die BPMI in die OMG integriert, wobei nur 2 Prozent der BPMI-Mitgliedsorganisationen das Standardisierungskonsortium verlassen haben. Die Arbeiten werden in der OMG vom BPMI Steering Commitee überwacht und im Rahmen der Business Mode­ ling & Integration (BMI) Domain Task Force (DTF) weiterentwickelt. Die aktuelle Version 1.0 der BPMN wurde im Jahr 2006 von der OMG verabschiedet. Zurzeit wird an einer umfassenden Überarbeitung des Standards gearbeitet, der in einer BPMN 2.0 münden soll (siehe Tab. 1).

458 citations


Book ChapterDOI
16 Jun 2008
TL;DR: The findings indicate that BPMN is used in groups of several, well-defined construct clusters, but less than 20% of its vocabulary is regularly used and some constructs did not occur in any of the models the authors analyzed.
Abstract: The Business Process Modeling Notation (BPMN) is an increasingly important industry standard for the graphical representation of business processes. BPMN offers a wide range of modeling constructs, significantly more than other popular languages. However, not all of these constructs are equally important in practice as business analysts frequently use arbitrary subsets of BPMN. In this paper we investigate what these subsets are, and how they differ between academic, consulting, and general use of the language. We analyzed 120 BPMN diagrams using mathematical and statistical techniques. Our findings indicate that BPMN is used in groups of several, well-defined construct clusters, but less than 20% of its vocabulary is regularly used and some constructs did not occur in any of the models we analyzed. While the average model contains just 9 different BPMN constructs, models of this complexity have typically just 4-5 constructs in common, which means that only a small agreed subset of BPMN has emerged. Our findings have implications for the entire ecosystems of analysts and modelers in that they provide guidance on how to reduce language complexity, which should increase the ease and speed of process modeling.

355 citations


Book ChapterDOI
02 Sep 2008
TL;DR: An approach for automated compliance checking of process activities and their ordering is introduced and a set of reduction rules are employed to address the problem of state-space explosion.
Abstract: Compliance rules describe regulations, policies and quality constraints business processes must adhere to. Given the large number of rules and their frequency of change, manual compliance checking can become a time-consuming task. Automated compliance checking of process activities and their ordering is an alternative whenever business processes and compliance rules are described in a formal way. This paper introduces an approach for automated compliance checking. Compliance rules are translated into temporal logic formulae that serve as input to model checkers which in turn verify whether a process model satisfies the requested compliance rule. To address the problem of state-space explosion we employ a set of reduction rules. The approach is prototypically realized and evaluated.

307 citations


Book ChapterDOI
Richard Hull1
09 Nov 2008
TL;DR: This short paper presents a structured framework for a class of data-centric business process models, which are based on "business artifacts", and identifies a broad array of remaining research challenges.
Abstract: A data-centric approach to business process and workflow modeling has been emerging over the past several years This short paper presents a structured framework for a class of data-centric business process models, which are based on "business artifacts" The paper provides a brief survey of research results on artifact-centric business process, and identifies a broad array of remaining research challenges

290 citations


Journal ArticleDOI
01 Jan 2008
TL;DR: A state-of-the-art review in the areas of business process modeling, analysis, and optimization-underlining that the latter two have not received enough coverage and support in the literature.
Abstract: There is an abundance of business process modeling techniques that capture and address different aspects of a business process. A limited number of these process models allow further quantitative analysis, and only a few enable structured process improvement. This paper reviews and classifies the main techniques for business process modeling with regard to their analysis and optimization capabilities. Three primary groups are identified, and a selection of representative business process modeling techniques is classified based on these. Similar classification is also presented for the analysis and optimization approaches for business processes that were identified in relevant literature. The main contribution of the paper is that it identifies which types of business process models are suitable for analysis and optimization, and also highlights the lack of such approaches. This paper offers a state-of-the-art review in the areas of business process modeling, analysis, and optimization-underlining that the latter two have not received enough coverage and support in the literature.

290 citations


Journal ArticleDOI
TL;DR: This paper discusses why context matters and how context can be conceptualised, classified and integrated with existing approaches to business process modelling and proposes a framework and a meta model for classifying relevant context.
Abstract: Flexibility has emerged as an important requirement in the design of business processes. Research on process flexibility, however, has traditionally been focused on the intrinsic capability of a process to adapt to a new environment (e.g. workflow escalation, ad hoc modelling). This paper proposes to extend the existing body of research by studying the extrinsic drivers for process flexibility, that is, the root causes that actually drive the demand for flexible business processes. The drivers for flexibility can be found in the context of a process and may include among others time, location, weather, legislation or performance requirements. We argue for a stronger and more explicit consideration of these contextual factors in the design of business processes in order to make processes more adaptive. This paper discusses why context matters and how context can be conceptualised, classified and integrated with existing approaches to business process modelling. We use a goal-oriented process modelling approach to be able to identify relevant context elements and propose a framework and a meta model for classifying relevant context. These extensions are an essential foundation for the definition and implementation of truly agile processes, and as such of high practical and theoretical value.

230 citations



Proceedings Article
01 Jan 2008
TL;DR: This paper presents a classification of definitions found in the IS literature, proposing guidelines on which to develop a more comprehensive definition in order to reach consensus, and identifies the four main business model concepts and values and their interaction, and thus place the business model within the world of digital business.
Abstract: Recent rapid advances in ICTs, specifically in Internet and mobile technologies, have highlighted the rising importance of the Business Model (BM) in Information Systems (IS). Despite agreement on its importance to an organization’s success, the concept is still fuzzy and vague, and there is no consensus regarding its definition. Furthermore, understanding the BM domain by identifying its meaning, fundamental pillars, and its relevance to other business concepts is by no means complete. In this paper we aim to provide further clarification by first presenting a classification of definitions found in the IS literature; second, proposing guidelines on which to develop a more comprehensive definition in order to reach consensus; and third, identifying the four main business model concepts and values and their interaction, and thus place the business model within the world of digital business. Based on this discussion, we propose a new definition for the business model that we argue is more appropriate to this new world.

196 citations


Book ChapterDOI
27 Oct 2008
TL;DR: This paper shows how a subset of the BPMN can be given a process semantics in Communicating Sequential Processes, which allows developers to formally analyse and compare BPMn diagrams.
Abstract: Business Process Modelling Notation (BPMN), developed by the Business Process Management Initiative (BPMI), intends to bridge the gap between business process design and implementation. However, the specification of the notation does not include a formal semantics. This paper shows how a subset of the BPMN can be given a process semantics in Communicating Sequential Processes. Such a semantics allows developers to formally analyse and compare BPMN diagrams. A simple example of a business process is included to demonstrate the application of the semantics; some theoretical results about the semantics are briefly discussed.

Journal ArticleDOI
TL;DR: Methods for identifying not only perfectly block-structured fragments in BPMN models, but quasi-structuring fragments that can be turned into perfectly structured ones and flow-based acyclic fragments that are mapped onto a combination of structured constructs and control links are defined.
Abstract: The Business Process Modelling Notation (BPMN) is a graph-oriented language in which control and action nodes can be connected almost arbitrarily. It is primarily targeted at domain analysts and is supported by many modelling tools, but in its current form, it lacks the semantic precision required to capture fully executable business processes. The Business Process Execution Language for Web Services (BPEL) on the other hand is a mainly block-structured language, targeted at software developers and supported by several execution platforms. In the current setting, translating BPMN models into BPEL code is a necessary step towards standards-based business process development environments. This translation is challenging since BPMN and BPEL represent two fundamentally different classes of languages. Existing BPMN-to-BPEL translations rely on the identification of block-structured patterns in BPMN models that are mapped into block-structured BPEL constructs. This paper advances the state of the art in BPMN-to-BPEL translation by defining methods for identifying not only perfectly block-structured fragments in BPMN models, but also quasi-structured fragments that can be turned into perfectly structured ones and flow-based acyclic fragments that can be mapped into a combination of block-structured constructs and control links. Beyond its direct relevance in the context of BPMN and BPEL, this paper addresses issues that arise generally when translating between graph-oriented and block-structured flow definition languages.

Journal ArticleDOI
01 Feb 2008
TL;DR: A formal two-step approach for constructing customized process views on structured process models by hiding and omitting activities from the non-customized view that are not requested by the process consumer is described.
Abstract: To enable effective cross-organizational collaborations, process providers have to offer external views on their internal processes to their partners. A process view hides details of an internal process that are secret to or irrelevant for the partners. This paper describes a formal two-step approach for constructing customized process views on structured process models. First, a non-customized process view is constructed from an internal structured process model by aggregating internal activities the provider wishes to hide. Second, a customized process view is constructed by hiding and omitting activities from the non-customized view that are not requested by the process consumer. The feasibility of the approach is shown by means of a case study.

Book ChapterDOI
16 Jun 2008
TL;DR: The goal of this paper is to formalize the information-centric approach to business process modeling and derive the relationships between the two approaches by formally defining the notion of a business entity from first principles and using this definition to derive an algorithm that generates an Information-centric process model from an activity-centric model.
Abstract: Most of the work in modeling business processes is activity-centric. Recently, an information-centric approach to business process modeling has emerged, where a business process is modeled as the interacting life cycles of information entities. The benefits of this approach are documented in a number of case studies. The goal of this paper is to formalize the information-centric approach and derive the relationships between the two approaches. We do this by formally defining the notion of a business entity from first principles and using this definition to derive an algorithm that generates an information-centric process model from an activity-centric model. We illustrate the two models using a real-world business process and provide an analysis of the respective strengths and weaknesses of the two modeling approaches.

Journal ArticleDOI
TL;DR: A non-conventional business process modeling method is introduced that is amenable to automatic analysis (simulation), yet powerful enough to capture the rich reality of business systems as enacted in the behavior and interactions of users.

Proceedings ArticleDOI
07 Jan 2008
TL;DR: The state of the art for flexible business process management systems and criteria for comparing them are presented.
Abstract: In competitive and evolving environments only organizations which can manage complexity and can respond to rapid change in an informed manner can gain a competitive advantage During the early 90's, workflow technologies offered a transversal integration capacity to the enterprise applications. Today, to "integrate " enterprise applications -and the activities they support- into business processes is not sufficient. The architecture of this integration should also be flexible. Enterprise requirements highlight flexible and adaptive processes whose execution can evolve (i) according to situations that cannot always be prescribed, and/or (ii) according to business changes (organizational, process improvement, strategic ...). More recent works highlight requirements in term of flexible and adaptive workflows, whose execution can evolve according to situations that cannot always be prescribed. This paper presents the state of the art for flexible business process management systems and criteria for comparing them.

Journal ArticleDOI
TL;DR: A mapping from Workflow Nets (WF-nets) to BPEL is provided, which builds on the rich theory of Petri nets and can also be used to map other languages onto BPEL.
Abstract: The Business Process Execution Language for Web Services (BPEL) has emerged as the de facto standard for implementing processes. Although intended as a language for connecting web services, its application is not limited to cross-organizational processes. It is expected that in the near future a wide variety of process-aware information systems will be realized using BPEL. While being a powerful language, BPEL is difficult to use. Its XML representation is very verbose and only readable for the trained eye. It offers many constructs and typically things can be implemented in many ways, e.g., using links and the flow construct or using sequences and switches. As a result only experienced users are able to select the right construct. Several vendors offer a graphical interface that generates BPEL code. However, the graphical representations are a direct reflection of the BPEL code and not easy to use by end-users. Therefore, we provide a mapping from Workflow Nets (WF-nets) to BPEL. This mapping builds on the rich theory of Petri nets and can also be used to map other languages (e.g., UML, EPC, BPMN, etc.) onto BPEL. In addition to this we have implemented the algorithm in a tool called WorkflowNet2BPEL4WS.

Patent
24 Jul 2008
TL;DR: A computer method and apparatus defining Business Classes for modeling business activities comprising the steps of representing business activities as the interaction between one or more Business Classes, and entering into a computer and storing in an electronic format the Business Classes and the relationships existing between the Business classes.
Abstract: A computer method and apparatus defining Business Classes for modeling business activities comprising the steps of representing business activities as the interaction between one or more Business Classes, and entering into a computer and storing in an electronic format the Business Classes and the relationships existing between the Business Classes.

Proceedings Article
Christopher J. Pavlovski1, Joe Zou1
01 Jan 2008
TL;DR: This work proposes how two new artifacts may be applied to model the constraints associated with a business process, and how these methods will assist in mitigating risk and facilitate the early discovery of non-functional requirements during systems development.
Abstract: Business process modeling entails the capture of a set of tasks that invariably model the functional behavior of a system. Another aspect of business process modeling involves the accurate capture of operational behavior and the associated process constraints. Whether the process is automated or manual, such operational constraints and behavior exist. This may include a variety of properties including performance expectations, policy constraints, and security controls. These characteristics later manifest as the non-functional requirements of an intended system, and often such information is generally identified at some point after the business process modeling exercise. The non-functional characteristics of the business are arguably more difficult to capture in business process modeling, since the focus of such methods is the modeling of functional behavior. We propose how two new artifacts may be applied to model the constraints associated with a business process. This is the operating condition to denote a business process constraint and the control case to define controlling criteria to mitigate risk associated with an operational condition. Modeling constraints in this way provides an opportunity to capture these characteristics of business process early in the systems development life-cycle. This contributes to a model that provides a more complete representation of the overall business process. The methods will assist in mitigating risk and facilitate the early discovery of non-functional requirements during systems development.

Book ChapterDOI
02 Sep 2008
TL;DR: This paper presents an approach where business users receive help in understanding the context and consequences of applying a pattern and describes an extension of a business process modeling tool with patterns to provide these capabilities.
Abstract: Although the business process community has put a major emphasis on patterns, notably the famous workflow patterns, only limited support for using patterns in today's business process modeling tools can be found. While the basic workflow patterns for control flow are available in almost every business process modeling tool, there is no support for the user in correctly applying these simple patterns leading to many incorrectly modeled business processes. Only limited support for pattern compounds can be found in some tools, there is no active support for selecting patterns that are applicable in some user-determined context, tools do not give feedback to the user if applying a pattern can lead to a modeling error, nor do they trace the sequence of applied patterns during the editing process. In this paper, we describe an extension of a business process modeling tool with patterns to provide these capabilities. We distinguish three scenarios of pattern application and discuss a set of pattern compounds that are based on the basic workflow patterns for control flow. We present an approach where business users receive help in understanding the context and consequences of applying a pattern.

Book ChapterDOI
23 Jun 2008
TL;DR: The establishment of the verification will be surely helpful toward solving the problems on business process reengineering, business process management, service-oriented architecture, and so on.
Abstract: Information systems have to respond well to the changing business environment. Thus, they must have architecture which withstands the change. To design such systems, business process modeling is effective, however, the models include often abstractness and arbitrariness. Therefore, there have been efforts that validate rigorousness of the models. They have defined semantics of the models and applied various logics and formal methods to verification of the rigorousness. This paper focuses on formal verification of the models and surveys the efforts. We also discuss the prospect of the solutions. The establishment of the verification will be surely helpful toward solving the problems on business process reengineering, business process management, service-oriented architecture, and so on.

Journal ArticleDOI
01 Jan 2008
TL;DR: To manage processes of realistic size, the problem of analyzing the interaction between WS-BPEL processes is addressed and a concept of a flexible model generation is presented which allows the generation of compact Petri net models.
Abstract: We address the problem of analyzing the interaction between WS-BPEL processes. We present a technology chain that starts out with a WS-BPEL process and translates it into a Petri net model. On the model we decide controllability of the process (the existence of a partner process, such that both can interact properly) and compute its operating guideline (a characterization of all properly interacting partner processes). To manage processes of realistic size, we present a concept of a flexible model generation which allows the generation of compact Petri net models. A case study demonstrates the value of this technology chain.

Book ChapterDOI
02 Sep 2008
TL;DR: In the academic business process management community, tooling plays an increasingly important role and the practical applicability of the research work can be demonstrated, which is important to raise awareness of academic BPM research to practitioners.
Abstract: In the academic business process management community, tooling plays an increasingly important role [8,6]. There are good reasons for this fact. Firstly, theoretical concepts benefit from exploration using prototypical implementation of the concepts. By experimentation based on real-world business processes, concepts can be evaluated and refined. Secondly, the practical applicability of the research work can be demonstrated, which is important to raise awareness of academic BPM research to practitioners.

Journal ArticleDOI
01 Nov 2008
TL;DR: A set of concepts and a methodology toward business process intelligence (BPI) using dynamic process performance evaluation, including measurement models based on activity-based management (ABM) and a dynamic enterprise process performance evaluated methodology are presented.
Abstract: Business process management systems traditionally focused on supporting the modeling and automation of business processes, with the objective of enabling fast and cost-effective process execution. As more and more processes become automated, customers become increasingly interested in managing process execution. This paper presents a set of concepts and a methodology toward business process intelligence (BPI) using dynamic process performance evaluation, including measurement models based on activity-based management (ABM) and a dynamic enterprise process performance evaluation methodology. The proposed measurement models support the analysis of six process flows within a manufacturing enterprise including activity flow, information flow, resource flow, cost flow, cash flow, and profit flow, which are crucial for enterprise managers to control the process execution quality and detect problems and areas for improvements. The proposed process performance evaluation methodology uses time, quality, service, cost, speed, efficiency, and importance as seven evaluation criteria. A prototype system supporting dynamic enterprise process modeling, analysis of six process flows, and process performance prediction has been implemented to validate the proposed methodology.

Book ChapterDOI
01 Jun 2008
TL;DR: This work presents a core ontology aimed at enhancing the state of the art in BPA and is structured around the process, resource, and object perspectives as typically adopted when analysing business processes.
Abstract: Business Process Management (BPM) aims at supporting the whole life-cycle necessary to deploy and maintain business processes in organisations. An important step of the BPM life-cycle is the analysis of the processes deployed in companies. However, the degree of automation currently achieved cannot support the level of adaptation required by businesses. Initial steps have been performed towards including some sort of automated reasoning within Business Process Analysis (BPA) but this is typically limited to using taxonomies. We present a core ontology aimed at enhancing the state of the art in BPA. The ontology builds upon a Time Ontology and is structured around the process, resource, and object perspectives as typically adopted when analysing business processes. The ontology has been extended and validated by means of an Events Ontology and an Events Analysis Ontology aimed at capturing the audit trails generated by Process-Aware Information Systems and deriving additional knowledge.

Proceedings ArticleDOI
12 May 2008
TL;DR: An approach to business process management, which leverages Agent Technology, especially BDI-Agent features to obtain agile business process behavior is presented.
Abstract: Business processes are the core assets of enterprises. They turn the business potential into actual competitiveness on the market. To face the challenges posed by today's changing and uncertain business environment, traditional business process management (BPM) approaches are not sufficient anymore. This paper presents an approach to business process management, which leverages Agent Technology, especially BDI-Agent features to obtain agile business process behavior. This paper sketches the problem, describes the solution approach, and presents the experiences gained in a concrete case study in the domain of Engineering Change Management.

Journal ArticleDOI
TL;DR: BP-QL as discussed by the authors is a query language for querying business processes based on an intuitive model of business processes, an abstraction of the emerging BPEL (business process execution language) standard.


Patent
24 Jan 2008
TL;DR: In this paper, the authors present methods and systems for developing, implementing, transforming, and governing a business model of an enterprise, which is based on interrelated business strategy, business goal and business constraint data.
Abstract: The present disclosure relates to methods and systems for developing, implementing, transforming, and governing a business model of an enterprise. In some embodiments, a method of implementing and managing a business model of an enterprise includes defining a business model using a business model tool that is executed by a computer, wherein the business model is based on interrelated business strategy, business goal and business constraint data, and includes a business service, inputting into the computer a first set of data representing a benchmark performance value of the business service, and operating the enterprise in accordance with the business model. The enterprise is monitored to determine an actual performance value of the business service, and the actual performance value and the benchmark performance value are processed in the computer to affect a comparison therebetween. The business model is modified based on the comparison.

Journal ArticleDOI
TL;DR: A metric is presented and described to analyze the control-flow complexity of business processes and is evaluated in terms of Weyuker’s properties in order to guarantee that it qualifies as good and comprehensive.
Abstract: Organizations are increasingly faced with the challenge of managing business processes, workflows, and recently, Web processes. One important aspect of business processes that has been overlooked is their complexity. High complexity in processes may result in poor understandability, errors, defects, and exceptions, leading processes to need more time to develop, test, and maintain. Therefore, excessive complexity should be avoided. Business process measurement is the task of empirically and objectively assigning numbers to the properties of business processes in such a way so as to describe them. Desirable attributes to study and measure include complexity, cost, maintainability, and reliability. In our work, we will focus on investigating process complexity. We present and describe a metric to analyze the control-flow complexity of business processes. The metric is evaluated in terms of Weyuker’s properties in order to guarantee that it qualifies as good and comprehensive. To test the validity of the metric, we describe the experiment we have carried out for empirically validating the metric.