1
Abstract— Since big data sets are structurally complex,
high-dimensional, and their attributes exhibit some redundant
and irrelevant information, the selection, evaluation, and
combination of those large-scale attributes pose huge challenges
to traditional methods. Fuzzy rough sets have emerged as a
powerful vehicle to deal with uncertain and fuzzy attributes in
big data problems that involve a very large number of variables
to be analyzed in a very short time. In order to further overcome
the inefficiency of traditional algorithms in the uncertain and
fuzzy big data, we present a new co-evolutionary fuzzy attribute
order reduction algorithm (CFAOR) based on a complete
attribute-value space tree. A complete attribute-value space tree
model of decision table is designed in the attribute space to
adaptively prune and optimize the attribute order tree. The fuzzy
similarity of multi-modality attributes can be extracted to satisfy
the needs of users with the better convergence speed and
classification performance. Then the decision rule sets generate a
series of rule chains to form an efficient cascade attribute order
reduction and classification with a rough entropy threshold.
Finally, the performance of CFAOR is assessed with a set of
benchmark problems that contain complex high dimensional
datasets with noise. The experimental results demonstrate that
CFAOR can achieve the higher average computational efficiency
and classification accuracy, compared with the state-of-the-art
methods. Furthermore, CFAOR is applied to extract different
tissues surfaces of dynamical changing infant cerebral cortex and
it achieves a satisfying consistency with those of medical experts,
which shows its potential significance for the disorder prediction
of infant cerebrum.
Index Terms— Complete attribute-value space tree,
Co-evolutionary fuzzy attribute order reduction, Rough entropy
threshold, Tissues extraction of infant cerebral cortex
I. INTRODUCTION
n recent years, with the development of various technologies,
a large number of data are being continuely generated
around us. Big data have attracted plenty of attention from a
This work was supported in part by the Australian Research Council (ARC)
under discovery grant DP180100656 and DP180100670, the National Natural
Science Foundation of China under Grant 61300167, Six Talent Peaks Project
of Jiangsu Province under Grant XYDXXJS-048, Jiangsu Provincial
Government Scholarship Program under Grant JS-2016-065, Natural Science
Foundation of Jiangsu Province under Grant BK20151274, and Applied Basic
Research Program of Nantong under Grant GY12016014. (Corresponding
author: Weiping Ding)
W. Ding is with the School of Computer Science and Technology, Nantong
University, Nantong 226019, China (e-mail: dwp9988@163.com)
I. Triguero is with the School of Computer Science, University of
Nottingham, United Kingdom (e-mail: Isaac.Triguero@nottingham.ac.uk)
C.-T. Lin is with the Centre for Artificial Intelligence, FEIT, University of
Technology Sydney, Ultimo NSW 2007, Australia (e-mail: chin-teng.lin
@uts.edu.au)
variety of fields such as biology, health, business management,
cognition analysis in human brain, and so on [1]-[3]. The data
contents include a complex mixture of texts, speeches, images,
and videos [4][5]. It is also true that massive amount of data
can potentially provide a much deeper understanding of both
nature and society, opening up new ways for research. It is,
however, a challenging task to extract useful knowledge from
such big data.
It has been observed that many real datasets are usually
structurally complex, high-dimensional, and multi-granular.
Those attributes usually exhibit some irrelevant and
redundancy information. In these cases, the big datasets
increase dynamically in size, which occur in a few of fields
such as the public health and welfare, economics analysis, and
medical research. Therefore, a series of emerging topics such
as big data acquisition, storage, management and processing
are important issues [6][7]. It becomes highly desirable to
develop some effective representative methodologies to
analyze big data and further handle their characteristics, such
as redundancy, uncertainty, fuzzy, and heterogeneity. The
massive amount of data makes traditional data analytical
methods inadequate to tackle many real-world high
dimensional problems. Although a large number of candidate
attribute sets is provided, most of them may turn out to be
redundant or irrelevant, which heavily deteriorates the
performance of traditional methods.
With the over-flooding of big data, researchers and
practitioners have started showing remarkable interest to
explore the data space, and have considered that structuralized
knowledge reasoning is an effective computational paradigm
for dealing with big data tasks. Granular computing (GC)
focuses on the knowledge representation and reasoning with
information granules, and fuzzy sets and rough sets are two
crucial branches of GC [8][9]. Fuzzy set theory (FST) was
introduced by Zadeh in 1965 to represent concepts with
ambiguous boundaries and to understand the processes of
complex human reasoning [10]. It has become a popular tool
for the design of fuzzy classifiers. However, a fuzzy set is only
characterized by a membership function, which largely ignores
the uncertainty. Rough set theory (RST) was presented by
Pawlak in 1982 to quantitatively analyze the uncertainty and to
process incomplete knowledge [11]. It can find a decision-
making table between the strict statistics and random
distribution. Since RST can typically describe the uncertainty
of knowledge, it has been extensively used in data mining,
knowledge discovery, and intelligent system [12]-[16]. Fuzzy
rough sets (FRS) appear as a combination of the advantages of
two complementary areas (RST and FST), which provides an
effective way to overcome the problem of discretization. It can
be widely applied to various kinds of attribute reduction
problems of numerical or continuous large-scale datasets
[17]-[20]. FRS is defined by two fuzzy sets, fuzzy lower and
Co-evolutionary Fuzzy Attribute Order Reduction
With Complete Attribute-value Space Tree
Weiping Ding, Member, IEEE, Isaac Triguero, and Chin-Teng Lin, Fellow, IEEE
I
2
upper approximations, which are obtained by extending the
corresponding crisp rough set notions. Elements that belong to
the lower approximation are considered to belong to the
approximated set with absolute certainty. Elements in the
fuzzy rough case have a membership in the range, which
allows for a greater flexibility in handling uncertain
information [21] [22]. Thus, there is a good potential to
improve reasoning and understanding of big data by using a
FRS method.
A. Related Work
In recent years, some significant algorithms and models
based on FRS have been presented. Zhao et al. [23] used one
generalized FRS model to construct a rule-based classifier, in
which the consistence degree was used for the reasonable
critical value to keep the discernibility information invariant.
The experimental results showed this model was effective and
feasible on noisy data, whereas, its computational capability in
big data needed to further be improved. Jensen and Cornelis
[24] exploited the concepts of lower and upper
approximations based on FRS and put forward a new nearest
neighborhood algorithm to classify all test objects and predict
their decision values. Experimental results showed that this
algorithm was competitive with some leading classification
methods. However, one obvious limitation was that no way
was designed to handle the data possessing missing values. Hu
et al. [25] summarized the properties of typical fuzzy rough
models in handling noisy tasks and revealed why they were
sensitive to the level of noise on fuzzy rough computation.
Then a collection of robust FRS models based on fuzzy lower
approximations is developed, and the experiments results on
real-world tasks illustrated the effectiveness of these models.
Parthaláin and Jensen [26] used FRS to select features for
inclusion and removal from the final candidate subset and
presented two unsupervised feature selection approaches as
UFRFS and dUFRFS. The approaches were shown to retain
useful features. But UFRFS utilized a simple but nevertheless
effective backwards elimination way for search, whilst
dUFRFS adopted a greedy forward selection way.
Furthermore, two search techniques often returned sub-
optimal results. Zeng et al. [27] combined the hybrid distance
and the Gaussian kernel to construct a novel FRS, and
presented the incremental algorithms for feature selection.
The efficiency for updating feature sets can be improved, but
the variation of multi-attributes was not taken into full
consideration. Maji and Garai [28] defined the lower and
upper relevance and significance of features for Interval type 2
(IT2) fuzzy approximation spaces, and presented an IT2
FRS-based attribute selection method by integrating the merits
of IT2 FRS and the maximal relevance-maximal significance
(MRMS) criterion. The effectiveness of the proposed method
was shown on several benchmarks and microarray gene
expression data sets. Yang et al. [29] presented two
incremental algorithms with FRS for attribute reduction in
terms of one incoming sample and multiple incoming samples,
respectively. The relative discernibility relation was
incrementally calculated for each condition attribute. The
experimental results demonstrated that proposed algorithms
could obtain the reduction result with the good classification
accuracy. But they were not applied to real-world big data. So
we need to further promote their efficiency on the complex,
high- dimensional, and multi-granular big data applications.
B. Limitations and Challenges
In the era of big data, the recent apparent progress of FRS
algorithms can be beneficial for the analysis of our confronting
big data problems. Meanwhile, it is worth mentioning that,
although these algorithms based on FRS are dominant in the
classification performance, there is still a lack of deep studies
for their applications in current complex big data. The
traditional algorithms are suffering from the essential
limitations and challenges as follows:
(1) Most of traditional algorithms are more suitable for the
medium datasets. If the sample size or the attribute size of the
datasets becomes very large, the processing time of attribute
reduction will tremendously grow with the increasing of
feature dimensions and number of instances. Furthermore, the
data dynamism is due to the mechanism that generates related
big data changes at different times or different real-world
circumstances, which adds new uncertainty for big data
analysis. Thus, that inherent interaction relation among
different attributes is not fully captured. Although we can
incorporate some known information about the desired data
partitions into decomposition process, it is not valid for
handling dynamic big data tasks. Improving the efficiency of
fuzzy rough attribute reduction algorithms in dynamically
increasing big data has become a significant research
topic,
which accelerates the process of finding reduction sets.
(2) The noise problem is one of the main sources of
uncertainty in big data applications. When adding noisy or
inconsistent data sets that have a lot of boundary objects, most
of traditional algorithms usually result in some undesirable
feature subsets since their auxiliary space will occupy a large
amount of memory, which will be detrimental to the attribute
reduction process. Moreover, data objects are normally
associated with complex classification scenarios. With the
dramatically increasing noise, the speed and volume
performances of data generation will deteriorate rapidly. So it
is very difficult to generate accurate fuzzy similarity relations
for the effective process of fuzzy attribute reduction. The
result is often unable to guarantee that the desired reduction
set is the optimal attribute set which satisfies the user’s need.
Consequently, the performance of attribute reduction is
oftenunreliable in most cases. Clearly, the noise problem in
big data greatly restricts the practical applications of
traditional algorithms.
C. Contributions
In order to address the challenges listed above, we present a
new co-evolutionary fuzzy attribute order reduction (CFAOR)
algorithm based on complete attribute-value space tree (CAST)
to develop the efficient attribute reduction performance for the
high-dimensional and uncertain big data. CAST of decision
table in the attribute space can adaptively prune and optimize
the attribute order tree. The reduced attribute set can satisfy
the needs of users with a better convergence speed, which
provides the same classification performance with the original
attribute set. This CFAOR algorithm is not only suitable for
the changing large-scale datasets with interdependent and
overlapping attribute variables, but also satisfies the
large-scale complex noisy data, which can preserve the
3
consistency of a given decision table. This CAST provides a
new viewpoint to understand and extend the FRS-based fuzzy
attribute reduction of big data.
CFAOR is widely compared with state-of-the-art fuzzy
attribute reduction methods on publicly-available datasets.
The experimental results demonstrate the superiority of
CFAOR. CFAOR is applied to identify different tissues
surfaces of dynamical changing infant cerebral cortex and it
can find more preferred different tissue surfaces from
dynamical changing infant cerebrum regions. These
encouraging results can achieve the satisfying consistency with
those of medical experts. So the main advantages of CFAOR
are the high efficiency and robustness for attribute reduction
solutions, which makes it particularly suitable for complex big
data.
D. Organization
This paper is organized as follows: In Section II, we provide
some preliminaries. A CAST model of decision table is
constructed in Section III. A new CFAOR algorithm is
presented in Section IV. An extensive experimental evaluation
is provided in Section V. The application performance in the
tissues extraction of dynamical changing infant cerebrum
regions is detailed in Section VI. Finally, some conclusions are
given in Section VII.
Provide an
application in
brain tissue
extraction
Conduct
extensive
experimental
evaluations
Preliminaries
Conclusions
Structure of
the paper
Contruct
CAST model
Present
CFAOR
algorithm
based CAST
Fig.1. The structure of the paper.
II. PRELIMINARIES
In this section, we introduce some relevant preliminaries
related to fuzzy rough set theory and decision-making system
with rough entropy threshold.
Definition 1 [10] Let
U
be a non-empty finite set of
samples. Each sample is described by a set of real-valued
condition attributes
A
and a symbolic decision attribute set
{}
D
d
. The pair
(, )UA D
and
AD
is called a fuzzy
decision table. If the decision attribute
d
divides
U
into a
family of disjoint subsets
{[ ] : }
D
UD x x U
,
[] { : () ()}
D
x
yUdx dy
is denoted as the decision class to
which the sample x belongs.
Definition 2 [8] Generalization of the granule based
approximation operators can be obtained by replacing the
partition
(/)UE
by a covering of
U
. Let
I
be an index
set, then a collection
{|}
i
K
Ui I
of non-empty
subsets of
U
is called a covering of
U
if
i
iI
KU
.
Definition 3 [30][31] A pair of approximation operators is
called as dual, if for all
, ( ()) ( ())
A
U apr co A co apr A
,where
()co
denotes a
covering operation of universe of objects. Equivalence classes
can be generalized by neighborhood operators. A
neighborhood operator
N
is a mapping
:()
N
UU
, where
()U
represents the collection of subsets of
U
. It is assumed
that the neighborhood operator is reflexive, i.e.,
()
x
Nx
for
all
x
U
.
Definition 4 [32] For each condition attribute
aA
, one can
define a fuzzy binary relation
a
R
, which is called a fuzzy
equivalence relation if
R
a is reflexive
(, ) 1Rxx
,
symmetric
(, ) (, )Rxy Ryx
, and sup-min transitive as
(, ) { (,), (, )}
zU
Rxy sup minRxz Rzy
,
,.
x
yU
(1)
A subset
BA
can also define a fuzzy equivalence
relation, denoted by
,
B
aB a
RR
(2)
where
a
R
is a fuzzy equivalence relation.
Let
()
F
U
be the fuzzy power set of
U
and
BA
. For
each
x
U
, a pair of lower and upper approximation
operators of
()
X
FU
based on
B
R
is defined as
()() {1 (,), ()},
BuUB
RXx inf max RxuXu
(3)
()() { (,),()}.
BuUB
RXx sup minRxuXu
(4)
()()
B
RXx
is considered as the degree of x certainly
belonging to
X
, while
()()
B
RXx
is the degree of x
possibly belonging to
X
.
(), ()
BB
RXRX
is referred to as
the fuzzy rough set of
X
with respect to
B
.
The essences of lower and upper approximations are
demonstrated in Fig. 2.
The most similar sample with
the same class to the given
sample,whose similarity
degree to the given sample is
the upper approximation value.
The nearest sample with
different class to the given
sample, whose distance to the
given sample is the lower
approximation value.
Fig.2. Diagrammatical representation of fuzzy rough approximation operator.
Definition 5 For a fuzzy decision table
(, )UA D
, and
BA
,
the fuzzy-rough positive region of
D
with respect to
B
is
defined as
4
() ().
BB
XUD
P
os D R X
(5)
Definition 6 An attribute subset
P
A
is called a reduct of
A
relative to
D
, if the following conditions are satisfied:
(i) For
(),
x
U
([])() ([])();
PD AD
Rx x Rx x
(6)
(ii) For
(),aP
yU
which satisfies with
{}
([ ] )( ) ([ ] )( )
Pa D A D
RyyRyy
. (7)
So a reduct
P
is a minimum attribute subset of condition
attributes that keeps the positive region of
D
with respect to
A
. It can discern these sample pairs, of which the
corresponding discernibility attribute sets are not empty.
Definition 7 For the fuzzy-rough attribute reduction process, it
must be able to find the dependency between various subsets
of the original feature set to deal with multiple features. It is
necessary to determine the degree of dependency of the
decision attribute with respect to
{,}
P
ab
. In the fuzzy
case, since objects may belong to several equivalence classes,
the cartesian product of
/({})UINDa
and
/({})UINDb
must be considered to determine
/UP
as follows:
/ { : / ({ })}.UP a PUINDa
(8)
Definition 8 For a fuzzy decision table
(, )UA D
and the
condition attribute set
12
{ , ,..., }
m
Aaa a
, the attribute order
satisfying the user’s requirements under
A
is denoted as
12
(): ...
k
SA a a a
. Therefore, the optimal reduction
model of attribute order is defined as follows:
() ( ( ))
S
F
xMaxfR
(9)
s.t.
()RgQ
,
( ( ), ( )),QASRAR
where
:() ()
g
AS AR
is any output reduction of attribute
sequence with satisfying the needs of users, and
R
is the
reduction of decision table
Definition 9 Supposed a fuzzy decision table
(, )UA D
,
QA
as a subset of condition attributes,
1
/{,,}
S
UQ X X
and
1
/{,,}
t
Ud Y Y
. Let the rough
entropy threshold be
(0.75 0.95)
. The decision
rules set with
is defined as follows:
(i) If a decision rule class
j
Y
(
1
j
t
) is absolutely rough
set with the indiscernible relation
Q
, then the rule set
Qd
is an absolute rough decision rule set with
.
(ii) If a decision rule class
j
Y
(
jt
) is relatively rough set
with the indiscernible relation
Q
, then the rule set
Qd
is a relatively rough decision rule set with
.
III. COMPLETE ATTRIBUTE-VALUE SPACE TREE MODEL
In this section, we present a new optimization model of
complete attribute-value space tree structure for fuzzy rough
attribute order reduction to find the optimal solution. This tree
can adaptively adjust the topological structure of attribute
complete tree, and it can successfully finish pruning and
optimizing the attribute order tree for the high-dimensional
and uncertain big data. The reduced attribute set can satisfy
the needs of users with maintaining a good diversity and a
high convergence speed, which provides the same
classification performance as the original attribute set.
Definition 10 Basic attributes-value tree denotes that each node
of tree is assigned an attribute associated with a basic category,
and each branch of node is assigned a value in the range of
node attribute’s value. The related attributes from the root node
to any leaf nodes are satisfied with the given related attribute
order.
Given a fuzzy decision table
(, )UA D
, the attributes order
()SB
is defined as
12
( ) : ...
k
SB a a a
, (
BA
,
12
{ , ,..., }
k
Baa a
). (10)
Attributes-value tree with
()SB
is a descending order
tree, in which each non-leaf node is given an attribute value
i
a
of
B
, each branch is assigned a value in the range of
i
a
,
and each node is associated with a subset of
U
.
Definition 11 Given a fuzzy decision table
(, )UA D
and
condition attribute subset
B
A
,
(, ( ), )TaSBU
represents
the complete attribute-value tree of
()SB
in
U
.
()
D
t
Td
is used to represent the corresponding leaf nodes
t
a
in
(, ( ), )TaSBU
, which can be also denoted as
(, ( ), )
D
TaSBU
.
So the subtree on the root node
it
a
is represented as
()
D
it
Ta
.
Definition 11 Supposed that complete attribute-value space
tree (CAST) can be represented as a
n-order subtrees
1
{ ,..., ,..., }
in
TTT
, as outlined in Fig. 3, where Subpop
i
, Par
i
,
and Elit
i
refer to the i
th
subpopulation, the i
th
parent node and
the i
th
elitist node in
i
T
, respectively,
This CAST can self-adapt the subpopulation sizes in
different subtrees and it is employed to capture the interacting
attribute order variables by exploiting deep correlation and
interdependency among interacting attributes order subsets of
big data.
Initially, all co-evolutionary particles are assigned in each
node of the original attribute-value space tree, and each inner
branch is regarded as a subpopulation with the same number of
nodes.
As depicted in Fig. 3, it contains two kinds of nodes.
One is the ordinary particle, denoted by white hollow dot ‘
’,
and the other one is the elitist particle, denoted by the black
entity dot ‘
’, which is the best children node in each subtree.
In order to select the best elitist in each subpopulation,
particles will be compared by their fitness in attribute-value
space tree. In each iteration, each
i
Par
in
i
Subpop
is
compared to
i
Elit
in
i
T
. If
()()
ii
f
Elit f Par
, this elitist
node of this CAST will be moved up and be exchanged with
the parent node. This procedure is continued until all elitists are
selected.
Due to excessively pursuing the elitists, CAST regardless
of selection directions easily results in the pruning and
optimization of attribute order tree into the opposite direction
to the sink. Thus, the length of CAST is increased. With the
5
extension operators, the adjacent elitists will be integrated to
reconstruct a unified elitist attribute-value space tree.
(, ( ), )TaSB U
1
()
D
Ta
()
D
i
Ta
()
D
n
Ta
1
ˆ
()w
ˆ
()w
i
ˆ
()w
n
1
ˆ
()w
ˆ
()w
i
ˆ
()w
n
1
ˆ
()w
ˆ
()w
i
ˆ
()w
n
Fig. 3. Construction processes of CAST framework.
The main construction processes of CAST are described as
follows:
Algorithm 1: Complete attribute-value space tree (CAST)
1. According to the attribute order of
()SB
, all nodes in the i
t
h
layer (
[1, ]ik
) are assigned attribute value
i
a
. The non-leaf
nodes in the same level take the same attribute and all leaf
nodes are placed in the (k+1)
th
layer.
2. Each node in the (i+1)
th
layer is associated with an equivalent
class as
(1) 1 2
/{ , ,..., }
ii
E
Uaa a
(1)
()
i
E
.
3. If the i
th
node in the i
th
layer is associated with the equivalence
class as
i
E
and the child nodes in the (i+1)
th
layer is
associated with the equivalence class as
(1)ip
E
, the associated
equivalent classes of the child nodes are mutually disjoint as
(1) (1)ii p i p
EE
.
4.
()
D
i
Ta
is a subtree on
(, ( ), )TaSBU
, and the rules of
trimming
()
D
it
Ta
from
(, ( ), )TaSBU
are pruned as follows:
(i)
Calculate
kpP
E
E
, where
P
E
is the equivalence
classes associated with all leaf nodes in
()
D
it
Ta
, and
remove all branches of
()
D
it
Ta
. So the node
it
a
is the
leaf node
k
a
of the associated equivalence class
k
E
.
(ii)
Replace
k
a
by the subtree
()
D
k
Td
according to
features of
(, ( ), )TaSBU
, and continue to prune the
corresponding subtree.
5. Adopt the truncated basis to optimize the underlying structure
of attribute order tree by
xw
n
ii
i
u
,
(11)
where
n
is the number of variables,
w
i
is the selected
loading vector and
i
u
is its corresponding coefficient.
6. Select the ‘best-n-basis’ of attribute order tree in the descending
normalized energy score as follows:
ˆ
()wwxxw
TT
ii i
,
(12)
where
12
[, ,..., ]x
dn
n
R
xx x
is the training data matrix
of branches of n-order subtrees.
7. Generate a set of reference vector
12
{ , ,..., }
n
where
12
( , ,..., ),
iii i
n
(13a)
1
01
, ,..., , 1,
n
ii
jj
j
H
HH H
(13b)
where
H
is the number of divisions set along each branch
in
i
T
.
8. Perform the assignment of the subtree neighbourhood
12
( ) { , ,..., }
T
Ei i i i
,
12
( , ,..., )
TT iT
is the closest vectors to
i
based on the Euclidean distance in
i
T
.
9. Adopt the Spearman rank correlation coefficient to calculate
the similarity between two nearest elitists by
2
2
1
()
(, )1 .
(1)
pp
n
ij
i
ij
p
Elit Elit
Sim
nn
Elit Elit
(14)
10. Refine the average shared similarity based on all pairwise
similarities by
1
11
2
(, ).
(1)
nn
ij
iji
SS Sim
nn
E
lit Elit
(15)
11. Reconstruct the complete attribute-value space tree
E
T
by the
elitist sets with the ‘best-n-basis’ of attribute order tree as
12
1
{ , ,..., }Elit
n
SS
= Elit Elit Elit
,
(16)
where
12
(, ,..., }
n
iii i
Elit Elit ElitElit
is the elitist sets of
i
T
for
the ‘best-n-basis’ from the top level basis
1
p
B
as the
p
projection vectors
12
( , ,..., )
p
ww w
.
IV. FUZZY ATTRIBUTE ORDER REDUCTION ALGORITHM
BASED ON
CAST
Since CAST structure of decision table in the attribute space
adaptively prunes and optimizes the attribute order tree, the
optimal attribute order reduction set can be achieved efficiently.
The fuzzy similarity of multi-modality attributes can be fully
extracted to satisfy the needs of users with the better
convergence speed and classification performance. In this
section, we present a co-evolutionary fuzzy attribute order
reduction (CFAOR) algorithm for large datasets, especially for
