Assessing the performance of biogas plants with multi-criteria and data envelopment analysis

TL;DR: An assessment of 41 agricultural biogas plants located in Austria to determine their relative performance in terms of economic, environmental, and social criteria and corresponding indicators suggests that MCDA, and the use of IRIS in particular, constitutes a useful approach that can be applied in a complementary way to DEA.

About: This article is published in European Journal of Operational Research.The article was published on 2009-09-16 and is currently open access. It has received 128 citations till now. The article focuses on the topics: Data envelopment analysis & Multiple-criteria decision analysis.

Summary

1. Introduction

• Over the last two decades, a growing environmental awareness has changed the focus of energy planning processes from an almost exclusive concern with cost minimization of supply-side options to the need of explicitly including multiple and potentially conflicting aspects, such as cost and environmental issues, in decision support models.
• The Kyoto Protocol, the EU Renewables Directive 2001/77/EC, and the European Biomass Action Plan are examples of ambitious political goals fostering the development of energy conversion technologies based on RES.
• With these intentions in mind, this paper also addresses the challenge of determining how MCDA methods can be used in the context of efficiency evaluation, trying to keep the spirit behind DEA, while being able to use MCDA’s capabilities of explicitly incorporating the preferences of a decision-maker, not necessarily in the form of trade-off restrictions.
• Uncertainty is an intrinsic characteristic of real-world problems arising from multiple sources of distinct nature.
• In Section 5 the findings from the analysis are discussed and some conclusions are drawn.

2.1. DEA

• The attainment of high levels of performance is a key issue for the success of every organization.
• DEA models use these inputs and outputs to compute an efficiency score for a given DMU when this particular DMU is compared with all the other DMUs considered.
• The weights are chosen by the LP model such that a DMU is ‘‘shown in its best light”, i.e., that its efficiency score is maximized.
• The CCR model was presented in the seminal work of Charnes et al. (1978).
• In fact, the inputs and outputs are not generally equally relevant and some preference information must be included in the analysis.

2.2. MCDA

• The above-mentioned considerations about DEA led us to envisage the use of MCDA models to perform efficiency evaluation.
• In assessing the performance of DMUs in which technical, economic and environmental aspects are at stake, it is often important to use known standards (or theoretical maxima) and efficiency profiles.
• The ELECTRE TRI method belongs to the ELECTRE family of multi-criteria methods developed by Bernard Roy and his co-workers (Roy, 1996).
• Also, a set of indifference (qj), preference (pj) and veto (vj) thresholds for each criterion j and reference profiles can be defined.
• Information about these parameter values can be provided through the introduction of intervals, linear constraints, or even sorting examples (which are translated into constraints on the parameters that guarantee that those example results are reproduced).

3. Case study

• In Austria, an effective promotion of renewable energy technologies has been pursued in recent years, driven by the need to achieve ambitious energy and climate policy goals.
• 1 Note that in 2006 a revised Green Electricity Act and Ordinance entered into force, with amended feed-in tariffs and budget restrictions (BGB1.
• As a result, many different technologies, design concepts, and specific applications occurred on the market, some of which were either not very productive, energy-efficient, or reliable.
• Due to the attractive feed-in tariffs granted, anaerobic digestion of energy crops currently mainly aims at the generation of electricity, and much less so at heat generation (or the feed-in of purified biogas into the natural gas grid, if available).

4.1. Description of the data and parameters used

• The DMUs considered are a representative set of energy crop digestion plants in Austria, aimed at covering the whole spectrum of existing plant types and operating conditions.
• Cooling, safe transport and appropriate storage were also scrutinized.
• The sampled installations are geographically well distributed over the country.
• The main groups of evaluation aspects at stake for assessing the efficiency of energy crop digestion plants are: (1) substrate provision, storage and pre-treatment; (2) biogas production (by means of anaerobic digestion); (3) net utilization of heat and electricity; (4) digestate handling and disposal; and (5) greenhouse gas (GHG) emissions.
• For further details on data collection see Braun et al. (2005, 2007) and Laaber et al. (2005), and for further details about the various inputs and outputs and DEA model specifications scrutinized see Madlener (2006).

4.2. DEA

• In the first DEA model considered in this paper, the authors have used substrate (i2) and labor (i1) as inputs and the amount of net electricity (o1) and external heat (o2) as outputs.
• See Scheel, 2001; Seiford and Zhu, 2002), and although the choice influences the results, there does not seem to be an undisputed ideal method to handle undesirable outputs (Dyson et al., 2001).
• A scale transla- Please cite this article in press as: Madlener, R. et al., Assessing the perf Journal of Operational Research (2008), doi:10.1016/j.ejor.2007.12.051 tion was used to account for the negative net values, another modeling option known to have an influence on the results (Lovell and Pastor, 1995).
• Keeping the same inputs and outputs presented here, the main implications of using the variable returns to scale version (BCC-O) would be to add DMU 15 to the set of efficient units, without GHG emissions, and adding DMUs 33 and 38 to the set of efficient units, with GHG emissions considered.

4.3. MCDA

• In the MCDA approach the objective was to identify groups of DMUs that could be assigned to different efficiency labels, rather than computing a precise efficiency score or deriving a complete ranking.
• Each plant has to be assigned to one of these ordered categories, according to the multiple evaluation criteria.
• Note that although this approach leads to a high number of indicators as the number of criteria increases, it mimics the spirit of DEA: to allow each DMU to be evaluated according to multiple indicators and to choose the most favorable indicators (within the constraints that the decision-maker may impose, as the authors will illustrate further below).
• DMUs in the best categories, several options can be envisaged: (1) to make the category bound more demanding; (2) to require the support of more than one indicator (e.g., the support of half of the indicators, as depicted in Fig. 4); and/or (3) to add some information about the relative power of the indicators.

4.5. Further analyses with IRIS

• The choice of the best and worst DMUs in the MCDA study was performed without making any distinction between the indicators.
• This implies that the importance of g2 (electricity/ODS) cannot be lower than the importance of g1 (electricity/labor).
• Optionally, the ELECTRE TRI models also allow incorporating veto thresholds, such that, for instance, a DMU that is classified as C1 according to a given indicator will not be able to reach category C4 in a multi-criteria evaluation.
• A form which is easily perceived by managers is to ask for intervals for some of the parameters (for instance, the weights), aimed at capturing information that is not precisely known but can be taken as bounded within some acceptable limits.
• The MCDA analysis may complement the DEA analysis by providing another perspective from which the conclusions of DEA may be either strengthened or weakened.

5. Discussion and conclusions

• DEA is a data-oriented approach that requires no a priori specification of the functional form of the production model converting inputs into outputs.
• Moreover, managerial preference information is often required, since inputs and outputs do not generally have the same importance in assessing the efficiency of operational units.
• This has been the main motivation for the use of MCDA techniques, in order to assess the extent by which these could overcome those characteristics of DEA, and what adaptations would be needed to improve the quality of the assessment.
• An authority certifying sustainable development practices may use this type of MCDA to label energy production plants according to their efficiency, taking into account the inputs they consume, the energy and other desirable outputs they produce, as well as greenhouse gas emissions and other undesirable outputs.
• DEA, on the other hand, can be particularly suited to identify DMUs with efficiency gaps relative to the state of the art, given the observed efficiency frontier.

Figures

Table 1 Descriptive statistics (N = 41)

Table 4 Classification subject to importance constraints

Fig. 1. Definition of categories C1, . . . ,Ck

Fig. 2. DEA results without GHG emissions (input

Frequently asked questions

Q1. What are the contributions in "Assessing the performance of biogas plants with multi-criteria and data envelopment analysis" ?

This paper performs an assessment of 41 agricultural biogas plants located in Austria to determine their relative performance in terms of economic, environmental, and social criteria and corresponding indicators. To be able to use IRIS while keeping the spirit behind DEA, the evaluation criteria were defined as different output/input efficiency ratios, and no information about criteria weights was introduced at the outset. The results suggest that MCDA, and the use of IRIS in particular, constitutes a useful approach that can be applied in a complementary way to