A new measure of consensus with reciprocal preference relations: The
correlation consensus degree
Teresa Gonz´alez-Arteaga
a
, Roc´ıo de Andr´es Calle
b
, Francisco Chiclana
c
a
PRESAD Research Group and Multidisciplinary Institute of Enterprise (IME), Faculty of Sciences
University of Valladolid, Spain
b
BORDA Research Unit, PRESAD Research Group and Multidisciplinary Institute of Enterprise (IME),
University of Salamanca, Spain
c
Centre for Computational Intelligence, School of Computer Science and Informatics, Faculty of Technology,
De Montfort University, Leicester, UK
Abstract
The achievement of a ‘consensual’ solution in a group decision making problem depends on
experts’ ideas, principles, knowledge, experience, etc. The measurement of consensus has been
widely studied from the point of view of different research areas, and consequently different
consensus measures have been formulated, although a common characteristic of most of them
is that they are driven by the implementation of either distance or similarity functions. In
the present work though, and within the framework of experts’ opinions modelled via recip-
rocal preference relations, a different approach to the measurement of consensus based on the
Pearson correlation coefficient is studied. The new correlation consensus degree measures the
concordance between the intensities of preference for pairs of alternatives as expressed by the
experts. Although a detailed study of the formal properties of the new correlation consensus
degree shows that it verifies important properties that are common either to distance or to
similarity functions between intensities of preferences, it is also proved that it is different to
traditional consensus measures. In order to emphasise novelty, two applications of the proposed
methodology are also included. The first one is used to illustrate the computation process and
discussion of the results, while the second one covers a real life application that makes use of
data from Clinical Decision-Making.
Keywords: Reciprocal preference relations, Consensus measure, Pearson correlation
coefficient, Concordance opinions measure, Correlation consensus degree.
1. Introduction
Consensus reaching is an important component in decision making processes, and indeed it
plays a key role in the resolution process of group decision making problems. One of the most
significant current discussion in consensus research concerns the measurement and achievement
of consensus from both a theoretical and applied points of view. On the one hand, establishing
and characterising different methodologies to measure consensus have been addressed from a
Social Choice perspective [1, 3, 13]. On the other hand, within the Decision Making Theory
framework, modelling group decision making problems in order to reach a higher level of cohe-
siveness has been managed successfully [15, 32, 34, 38, 39, 65]. Outside of these main areas, it
is possible to find other methodologies that use the idea of consensus in different ways to the
aforementioned ones, with [41, 46] being representative examples of these methodologies.
Email addresses: teresag@eio.uva.es (Teresa Gonz´alez-Arteaga), rocioac@usal.es (Roc´ıo de Andr´es
Calle), chiclana@dmu.ac.uk (Francisco Chiclana)
Preprint submitted to Knowledge-Based Systems. Revised version of KNOSYS-D-15-01671. May 31, 2016
Despite the productive research on this area, consensus measurement is still an open-ended
research question because the methodology to use in each case is an essential component of
the problem. Up to now most studies on consensus measurement have focused on the use of
distance/similarity function based measures and association measures, respectively. Among the
distance functions used, and worth highlighting, are the Kemeny, Mahalanobis, Mannhattan,
Jacard, Dice and Cosine distance functions [1, 4, 6, 17, 19, 29, 31]. Association measures are
less widely used than distance functions but it is also possible to find the use of some of them
such as the Kendall’s coefficient, the Goodman-Kruskal’s index and the Spearman’s coefficient
[18, 24, 35, 44, 58].
In this paper we focus on establishing a new consensus measure following the tradition of
association measures. Our proposal is based on the original statistical correlation concept, the
Pearson correlation coefficient. Therefore, this new measure is an alternative to the use of
the aforementioned approaches. The Pearson correlation coefficient plays an important role
in Statistics and Data Analysis and it is extensively used as a measure of the degree of linear
dependence between two variables. It is easy to interpret as well as invariant to certain changes
in the variables [52, 55, 57]. Specifically, in this paper the notion of dependence among elements
from correlation coefficient as a measure of the cohesiveness between opinions is adopted. This
seems natural because the measurement of consensus resembles the notion of a “measure of
statistical correlation”, in the sense that the maximum value 1 captures the notion of unanim-
ity as a perfect relationship among agents’ preferences (experts’ preferences follow the same
direction), while the minimum value −1 captures the notion of total disagreement (experts’
preferences present a negative relationship). Furthermore, the higher the cohesiveness between
experts’ preferences, the more positive correlated the preferences are. Similarly, the lower the
cohesiveness between experts’ preferences, the more negative correlated the preferences are.
This new consensus measure will be developed within assumptions of experts’ opinions or
preferences being expressed by means of reciprocal preference relations, a framework that is cur-
rently of interest to the research community in decision theory under uncertainty [7, 27, 28, 45].
Under reciprocal preference relations, on the one hand and as it was mentioned above, the
new proposed approach inherits advantages of previous approaches based on traditional dis-
tance/similarity and association measures. On the other hand, maximum consensus tradition-
ally represents the case when experts provide the same preference intensities for each possible
pair of alternatives. This, though, is not the only possible scenario of maximum consensus.
Indeed, the proposal here put forward addresses this issue satisfactorily because maximum
possible cohesiveness or consensus between experts’ opinions does not necessary imply that
all reciprocal preference relations have to coincide, and therefore all experts do not necessary
need to have the same preference intensities in all possible pairs of alternatives. It is sufficient,
though, that experts rank alternatives in the same way. To support all these claims, a set of
properties verified by the new proposed measure of consensus, the correlation consensus de-
gree, are proved. These properties ensure the suitability of the correlation consensus degree.
Furthermore, in order to emphasise novelty, two applications of the proposed methodology are
also included. The first one is used to illustrate the computation process and discussion of the
results, while the second one covers a real life application that makes use of data from Clinical
Decision-Making.
The rest of the paper is organised as follows. Section 2 contains a brief overview of the
different approaches in literature to measure group cohesiveness. The basic notation and pre-
liminaries are presented in Section 3. Section 4 provides the new approach to consensus mea-
surement based on the Pearson correlation coefficient. In Section 5, properties of the new
correlation consensus degree are studied. Section 6 presents two practical applications of the
proposed methodology. Finally, some concluding remarks and future research are presented in
Section 7.
2
2. Consensus measurement in the literature
A considerable amount of literature has been published on measuring and reaching consensus
in group decision making problems. Consensus measurement is a prominent and active research
subject in several areas such as Social Choice Theory and Decision Making Theory. A brief
overview of how this issue has been addressed in recent literature from the aforementioned
research areas is provided.
From the Social Choice Theory, the first serious discussions and analysis of consensus mea-
surement from an Arrovian perspective emerged with Bosch’s PhD Thesis [13], where both
absolute and intrinsic measures of consensus were proposed, analysed and axiomatically char-
acterised. From the point of view of considering consensus among a family of voters, McMorris
and Powers [48] characterised consensus rules defined on hierarchies, while Garc´ıa-Lapresta
and P´erez-Rom´an [29] focused on how to measure consensus using complete preorders on al-
ternatives and introduced a class of consensus measures based on seven well-known distances.
Subsequently, Alcalde-Unzu and Vorstatz in [1] characterised a family of linear and additive
consensus measures, whereas in [2] new ways to measure the similarity of preferences in a
group of individuals were suggested. Alcantud, de Andr´es Calle and Casc´on [3] studied and
characterised a class of consensus measure, called referenced consensus measure, that permits
to produce a numerical social evaluation from purely ordinal individual information. This
measure has to be specified by means of a voting mechanism and a measure of agreement
between profiles of orderings and individual orderings. Moreover, Alcantud, de Andr´es Calle
and Casc´on in [5] contributed to the formal and computational analysis of the aforementioned
referenced consensus measure by focusing on two relevant and specific cases: the Borda and
the Copeland rules under a Kemeny-type measure. There are, however, situations where each
member of a population classifies a list of options as either acceptable or non-acceptable; either
agree or disagree, etc., and therefore generating a dichotomous preference structure. Under
this assumption, Alcantud, de Andr´es Calle and Casc´on [4] proposed the concept of approval
consensus measure and gave axiomatic characterisations of two generic classes of such approval
consensus measures. Alcantud, de Andr´es Calle and Gonz´alez-Arteaga [6] introduced the use of
the Mahalanobis distance for the analysis of the cohesiveness of a group of complete preorders
and proved that arbitrary codifications of the preferences are incompatible with their formula-
tion although affine transformations permit to compare profiles on the basis of such proposal.
Finally, it is worth mentioning a distance-based approach to measure the degree of consensus
considering approval information about alternatives as well as the rankings of them suggested
by Erdamar et al. in [25].
From the Decision Making Theory, a considerable amount of contributions have been made
since the 1980’s. As such, it is worth mentioning the first preliminary work on reaching consen-
sus and its measurements carried out by Kacprzyk and Fedrizzi [42], in which the concept of
“degree of consensus” in the sense of expressing the degree to which “most of” the individuals
in a group agree to “almost all of” the options. The point of departure of this paper being that
the experts’ opinions are expressed by fuzzy preference relations. Within this framework of
preference representation, different consensus measurement based on similarity measures have
been put forward by Herrera-Viedma, et al. [37] and Wu and Chiclana [63] for both com-
plete and incomplete information environments. The case when experts’ opinions are expressed
by means of linguistic assessments has been extensively studied and it is worth mentioning
the works of Ben-Arieh and Chen [12], Cabrerizo, Alonso and Herrera-Viedma [14], Garc´ıa-
Lapresta, P´erez-Rom´an [30], Herrera, Herrera-Viedma and Verdegay [36], Herrera-Viedma, et
al. [40], P´erez-Asurmendi and Chiclana [53] and Wu, Chiclana and Herrera-Viedma [65]. Fi-
nally, models to reach consensus where experts assess their preferences using different preference
representation structures (preference orderings, utility functions, multiplicative preference rela-
tions and fuzzy preference relations) have also been studied and proposed by Dong and Zhang
3
[23], Fedrizzi et al. [26] and Herrera-Viedma, Herrera and Chiclana [39]. The problem of
measuring and reaching consensus with intuitionistic fuzzy preference relations and triangular
fuzzy complementary preference relations have also been covered in detail by Wu and Chiclana
in [62, 64].
To conclude, Table 1 summarises and classifies the approaches that have been reviewed in
this Section.
Consensus measures in Social Choice Theory
Author(s)/Year Framework Measurement methodology
Bosch [13], 2005 Ordinal Inf.
Based on different distances
McMorris and Powers [48], 2009 Ordinal Inf.
Garc´ıa-Lapresta and P´erez-Rom´an [29], 2011 Ordinal Inf.
Alcalde and Vorsatz [1], 2013 Ordinal Inf.
Alcantud, de Andr´es Calle and Casc´on [3] [5], 2013 Ordinal Inf.
Alcantud, de Andr´es Calle and Casc´on [4], 2013 Dichotomous Inf.
Alcantud, de Andr´es Calle and Gonz´alez-Arteaga [6], 2013 Ordinal Inf.
Erdamar, et al. [25], 2014 Ordinal Inf.
Alcalde and Vorsatz [2], 2015 Ordinal Inf.
Consensus measures in Decision Making Theory
Author(s)/Year Framework Measurement methodology
Kacprzyk and Fedrizzi [42], 1988 Fuzzy Inf.
Based on collective solution
Fedrizzi et al. [26], 2010 Fuzzy Inf.
Herrera-Viedma et al. [37], 2007 Incomplete Fuz. Inf.
Herrera, Herrera-Viedma and Verdegay [36], 1996 Linguistic Inf.
Herrera-Viedma et al. [40], 2005 Linguistic Inf.
Cabrerizo, Alonso and Herrera-Viedma [14], 2009 Linguistic Inf.
Wu and Chiclana [62–64], 2014 Incomplete Fuz. and Ling. Inf.
Garc´ıa-Lapresta, P´erez-Rom´an and Falc´o [30], 2015 Linguistic Inf.
Wu, Chiclana and Herrera-Viedma [65], 2015 Incomplete Linguistic Inf.
Herrera-Viedma, Herrera and Chiclana [39], 2002 Different Inf.
Based on individual solution
Ben-Arieh and Chen [12], 2006 Linguistic Inf.
Dong and Zhang [23], 2014 Different Inf.
Table 1: Summary table of studies related to consensus measures
3. Preliminaries
This Section briefly presents the main concepts needed to make the paper self-contained,
and as such a short review of the terminology and the concept of fuzzy binary relation are
presented. The interested reader is advice to consult the following [7–9, 27, 28, 45, 50, 60].
Definition 1. Let X be a non empty set. A fuzzy binary relation P on X is a fuzzy subset of
the Cartesian product X × X characterised by its membership function µ
P
: X × X −→ [0, 1],
where µ
P
(x
1
, x
2
) = p
ij
represents the strength of the relation between x
1
and x
2
.
Henceforth, X is a finite set X = {x
1
. . . , x
n
} (n > 2), whose elements will be referred to
as alternatives. Abusing notation, on occasions alternative x
i
will be represented simply as i
for convenience.
Definition 2. A reciprocal preference relation on X is a fuzzy binary relation P where
µ
P
(x
i
, x
j
) = p
ij
∈ [0, 1] represents the partial preference intensity of element i over j and
that verifies the following property: p
ij
+ p
ji
= 1 ∀x
i
, x
j
∈ X.
In order to realise the meaning of a reciprocal preference relation, we suppose the following
common situation: an expert compares two alternatives x
i
and x
j
. In this specific context, the
4
expert not only establishes that the alternative x
i
is preferred to the alternative x
j
, but also
shows her/his intensity of preference between them by means of the value p
ij
. So, the higher
p
ij
, the higher the preference intensity of alternative x
i
over alternative x
j
. Thus, 0 < p
ij
< 0.5
would indicate that x
j
is preferred to x
i
. If p
ij
= 0.5 then alternatives x
i
and x
j
are equally
preferred. When 0.5 < p
ij
< 1, x
i
is preferred to x
j
. Moreover, p
ij
= 0 (resp. p
ij
= 1) indicates
that x
j
(resp. x
i
) is absolutely preferred to x
i
(resp. x
j
).
Let P be an n × n matrix that contains all the partial intensity degrees of a reciprocal
preference relation on the set X:
P =
p
11
p
12
··· p
1n
p
21
p
22
··· p
2n
.
.
.
.
.
.
.
.
.
.
.
.
p
n1
p
n2
··· p
nn
,
verifying 0 ≤ p
ij
≤ 1; p
ij
+ p
ji
= 1 for i, j ∈ {1, . . . , n}. The set of all these matrices n × n
is denoted by P
n×n
. Here it is also noticed that a reciprocal preference relation can also be
mathematically represented by means of a vector, namely the essential vector of preference
intensities.
Definition 3. The essential vector of preference intensities, V
P
, of a reciprocal preference
relation P = (p
ij
)
n×n
∈ P
n×n
is the vector made up with the
n(n − 1)
2
elements above its main
diagonal:
V
P
=
p
12
, p
13
, . . . , p
1n
, p
23
, . . . , p
2n
, . . . , p
(n−1) n
=
=
v
1
, . . . , v
r
, . . . , v
n(n−1)/2
.
The reciprocity property of reciprocal preference relations allows the alternative definition of
the essential vector of preference intensities of a reciprocal preference relation
as the vector composed of the preference values below the main diagonal,
V
P
t
= (p
21
, p
31
, . . . , p
n1
, p
32
, . . . , p
n2
, . . . , p
n (n−1)
).
4. A novel measurement of consensus based on the Pearson correlation coefficient
Based on the concept of correlation, specifically the Pearson correlation coefficient, this
section introduces a new consensus measure for group decision making problems under reci-
procal preference relations. First, we recall such a correlation coefficient and its properties as
necessary to define the new correlation consensus degree and associated properties.
4.1. Pearson correlation coefficient
The measurement of the relationship strength among variables is an important issue in
Statistical Analysis, and the Pearson correlation coefficient is a traditional tool used for that
purpose [52, 55].
Definition 4. Given a sample of n pairs of real values {(x
1
, y
1
), . . . , (x
n
, y
n
)}, the Pearson
correlation coefficient of the two n-dimensional vectors x = (x
1
, . . . , x
n
) and y = (y
1
, . . . , y
n
),
cor(x, y), is computed as
cor(x, y) =
n
X
i=1
(x
i
− x)(y
i
− y)
v
u
u
t
n
X
i=1
(x
i
− x)
2
v
u
u
t
n
X
i=1
(y
i
− y)
2
5
![Table 2: Control Preference Scale (CPS) [21]](/figures/table-2-control-preference-scale-cps-21-2kajcl3z.webp)
