Learning and Convergence of Fuzzy Cognitive Maps Used in Pattern Recognition

TLDR: A population-based learning algorithm with convergence features for FCM-based systems used in pattern classification, based on a heuristic procedure, called Stability based on Sigmoid Functions, which allows improving the convergence of sigmoid FCM used inpattern classification.

Abstract: In recent years fuzzy cognitive maps (FCM) have become an active research field due to their capability for modeling complex systems. These recurrent neural models propagate an activation vector over the causal network until the map converges to a fixed-point or a maximal number of cycles is reached. The first scenario suggests that the FCM converged, whereas the second one implies that cyclic or chaotic patterns may be produced. The non-stable configurations are mostly related with the weight matrix that defines the causal relations among concepts. Such weights could be provided by experts or automatically computed from historical data by using a learning algorithm. Nevertheless, from the best of our knowledge, population-based algorithms for FCM-based systems do not include the map convergence into their learning scheme and thus, non-stable configurations could be produced. In this research we introduce a population-based learning algorithm with convergence features for FCM-based systems used in pattern classification. This proposal is based on a heuristic procedure, called Stability based on Sigmoid Functions, which allows improving the convergence of sigmoid FCM used in pattern classification. Numerical simulations using six FCM-based classifiers have shown that the proposed learning algorithm is capable of computing accurate parameters with improved convergence features.

Figures

Table 1. Features of each learning problem adopted for evaluation.

Table 2. Results achieved by the Wilcoxon test.

Figure 1. Activation value of the decision neuron each iteration step for two stable configurations. The dashed line denotes the numerical response after applying the standard learning approach, whereas the solid line represents the numerical response after applying the proposed learning algorithm.

Figure 3. Activation value of the decision neuron each iteration step for two chaotic configurations. The dashed line denotes the numerical response after applying the standard learning approach, whereas the solid line represents the numerical response after applying the proposed learning algorithm.

Figure 2. Activation value of the decision neuron each iteration step for two cyclic configurations. The dashed line denotes the numerical response after applying the standard learning approach, whereas the solid line represents the numerical response after applying the proposed learning algorithm.

Full text

Made available by Hasselt University Library in https://documentserver.uhasselt.be
Learning and Convergence of Fuzzy Cognitive Maps Used in Pattern Recognition
Peer-reviewed author version
NAPOLES RUIZ, Gonzalo; PAPAGEORGIOU, Elpiniki; Bello, Rafael & VANHOOF,
Koen (2016) Learning and Convergence of Fuzzy Cognitive Maps Used in Pattern
Recognition. In: NEURAL PROCESSING LETTERS, 45 (2), pag. 431-444.
DOI: 10.1007/s11063-016-9534-x
Handle: http://hdl.handle.net/1942/22971

Learning and convergence of Fuzzy Cognitive
Maps used in pattern recognition
Gonzalo Nápoles
1,2,
*
, Elpiniki Papageorgiou
3,1
, Rafael Bello
2
Koen Vanhoof
1
1
Faculty of Business Economics, Hasselt University, Belgium
2
Department of Computer Sciences, Central University of Las Villas, Cuba
3
Department of Computer Engineering, Technological Education Institute of Central Greece, Greece
Abstract. In recent years Fuzzy Cognitive Maps (FCM) have become an active research field due to their
capability for modeling complex systems. These recurrent neural models propagate an activation vector
over the causal network until the map converges to a fixed-point or a maximal number of cycles is reached.
The first scenario suggests that the FCM converged, whereas the second one implies that cyclic or chaotic
patterns may be produced. The non-stable configurations are mostly related with the weight matrix that
defines the causal relations among concepts. Such weights could be provided by experts or automatically
computed from historical data by using a learning algorithm. Nevertheless, from the best of our knowledge,
population-based algorithms for FCM-based systems do not include the map convergence into their learning
scheme and thus, non-stable configurations could be produced. In this research we introduce a population-
based learning algorithm with convergence features for FCM-based systems used in pattern classification.
This proposal is based on a heuristic procedure, called Stability based on Sigmoid Functions, which allows
improving the convergence of sigmoid FCM used in pattern classification. Numerical simulations using six
FCM-based classifiers have shown that the proposed learning algorithm is capable of computing accurate
parameters with improved convergence features.
Keywords. Fuzzy Cognitive Maps, learning algorithm, convergence.
*
Corresponding author: gonzalo.napoles@uhasselt.be

I. Introduction
Fuzzy Cognitive Maps (FCM) are Recurrent Neural Networks for modeling dynamical systems
using causal relations [1]. Essentially, a FCM involves an information network where graph nodes
represent objects, states, concepts or entities of the investigated system and they comprise a precise
meaning for the problem domain. These concepts are equivalent to neurons in neural models, and
they are connected by causal relationships that take values in the range
󰇟

󰇠
. These elements
interact during the inference stage to update the activation value of each neuron by using a rule
similar to the standard McCulloch-Pitts schema [2]. This updating procedure is iteratively repeated
until (i) the FCM-based system converges to a fixed-point attractor or (ii) a maximal number of
iterations is reached. The former implies that a hidden pattern was discovered [3] whereas the latter
suggests that the system responses are cyclic or completely chaotic.
The non-stable configurations are mostly related with the causal weight matrix that describes the
whole system. More explicitly, a perfectly symmetric weight matrix implies the existence of large
number of positive cycles in the modeled system. These cycles provide the system with positive
feedback loops that amplify any initial change and thus lead to exponential growth or decline [4].
On the other hand, antisymmetric causal weight matrixes imply the existence of negative cycles
with odd number of connections, providing the FCM with negative feedback loops that counteract
any stimulus. Thus, after time period equal to the length of the cycle the neuron to which the initial
change was introduced will receive an influence that has an opposite sign from the initial change.
This leads the system to periodic behavior and the creation of limit cycles.

Such weights can be provided by domain experts or automatically computed from historical data
by using a learning algorithm. Existing learning methods can be grouped into two large groups:
Hebbian-based and population-based algorithms [5]. The first ones only require a single instance
to adjust the model, however, numerical experiments reported by Papakostas et al. [6] have shown
that population-based learning algorithms are preferred when developing FCM-based classifiers.
Unfortunately, these algorithms do not include any convergence feature into their learning scheme
and therefore, estimated parameters could induce non-stable behaviors.
Another challenging research field is related to the development of accurate FCM-based classifiers
since they often show lower prediction rates regarding to traditional classifiers (e.g., decision trees,
neural networks, support vector machines). However, in contrast to FCM-based models, traditional
classifiers perform like black-boxes and therefore they are difficult to interpret. Roughly speaking,
a FCM-based classifier can work in two types of architectures [6]:
 Class-per-output architecture. Each decision class is mapped as an output neuron. During
the exploitation of the FCM-based classifier, the predicted decision class corresponds to
the output neuron with the highest activation value.
 Single-output architecture. Each decision class is enclosed into the activation space of the
decision neuron. By doing so, two possibilities have been identified:
a) Using a clustering approach. During the training phase, the center of each cluster
is determined and labeled. In the testing phase, the center having the closest
distance to the projected activation value is assigned to the input pattern.
b) Using a thresholding approach. During the training phase, a pair of thresholds for
each decision class are determined. In the testing phase, the interval comprising the
projected activation value is assigned to the input pattern.

From the best of our knowledge, only a few studies addressing the convergence on FCM-based
classifiers have been proposed. For example, Boutalis et al. [7] and Kottas et al. [8] investigated
the existence and uniqueness of equilibrium values of neurons in FCM equipped with sigmoid
transfer functions, using the contraction mapping theorem. Knight et al. [9] proposed a slightly
different theoretical result related with the inclination of the sigmoid function. However, Nápoles
et al. [10] numerically verified that these theoretical results cannot be directly used in solving
pattern classification problems since a FCM-based classifier with a single fixed point-attractor will
produce the same decision class for all input patterns.
In this paper we introduce a population-based learning algorithm that attempts to compute accurate
parameters (i.e., the causal weights that define the interaction among map neurons, and the sigmoid
inclination of each transfer function) having convergence features. It implies that the FCM-based
classifier must be capable of effectively recognizing the input patterns in a stable fashion, that is,
reducing the variability on the responses for consecutive iterations. To accomplish that, we extend
the basic principle of a heuristic algorithm called Stability based on Sigmoid Functions (SSF) that
allows improving the convergence of FCM-based classifiers [10] [11]. It should be mentioned that
the proposed learning algorithm provides high flexibility and allows computing the parameters of
FCM-based classifiers having different decision architectures.
The rest of the paper is organized as follows: in Section II the background about the FCM theory
is provided, whereas in Section III we describe the SSF algorithm. In Section IV we introduce the
proposed algorithm to compute the causal weights and the sigmoid parameters in a stable fashion,
including some important definitions and theorems. Section V provides numerical simulations that
allow evaluating our learning methodology across six FCM-based classifiers, whereas in the last
section we discuss relevant remarks and further research aspects.

Frequently asked questions

Q1. What contributions have the authors mentioned in the paper "Learning and convergence of fuzzy cognitive maps used in pattern recognition" ?

In this research the authors introduce a populationbased learning algorithm with convergence features for FCM-based systems used in pattern classification. 

Q2. What are the future works in "Learning and convergence of fuzzy cognitive maps used in pattern recognition" ?

As a future work, the authors will be focused on hybridizing the proposed learning algorithm with existing learning rules for neural networks. 

Q3. What is the widely accepted divergence measure?

A widely accepted divergence measure is the Kullback–Leibler divergence [27] between the true model and the approximating candidate model. 

Q4. What is the significance degree of the Wilcoxon signed rank test?

The Wilcoxon signed rank test is a nonparametric method employed in hypothesis testing situations, involving a design with two samples. 

Q5. How many swarm particles are set to 80?

As mentioned, to minimize the error function (6) the authors adopt the constricted PSO Type-1 introduced by Clerk and Kennedy [23] as numerical optimizer, where the number of swarm particles is set to 80, the inertia weight 𝜔 = 0.7298 and 𝑐1 = 𝑐2 = 1.496. 

Q6. What is the heuristic search approach used in this study?

In their study the authors adopt an heuristic search approach since population-based metaheuristics are capable of finding near-optimal solutions in a reasonable execution time, thus ignoring analytical properties of the target function (e.g., convexity, continuity, differentiability or gradient information). 

Q7. What is the function that computes the prediction error?

Equation (6) shows the error function to be optimized, where 𝑋 is the candidate solution generated by the selected optimizer, 𝐾 denotes the number of training patterns, 0 ≤ 𝐹(. ) ≤ 1 is a function that computes the prediction error achieved by the classifier, whereas 0 ≤ 𝐻(. ) ≤ 1 represents the accumulated convergence error during updating the activation value of neurons. 

Q8. How many sequence positions were used in the proposed classifier?

Such sites were biologically determined from clinic assays in infected patients and they allowed an averaged reduction rate of 80% regarding the total number of sequence positions. 

Q9. What is the mapping of the input and output neurons?

In the second step, the mapping ℳ: [0,1]𝑁 → [0,1]𝑀−𝑁 corresponds to the updating rule that computesthe activation value of output neurons. 

Q10. What is the definition of a heuristic procedure for a non-discrete?

In this section the authors describe a heuristic procedure called Stability based on Sigmoid Functions (SSF) for non-discrete FCM-based systems [10] that allows improving the system convergence without altering the weights configuration. 

Q11. What is the expected resistance class for the inhibitor?

each training pattern comprises the activation value of the 𝑁 input neurons, and the expected resistance class for the inhibitor (i.e., 0-susceptible and 1-resistant). 

Q12. What is the function that determines the predicted class label?

Observe that the responses at the previous discrete-time steps are not considered since they are not used when computed the predicted class, instead, such responses are evaluated when analyzing the convergence of the FCM-based classifier. 

Q13. What is the description of the learning method?

More importantly, during simulations the authors observed that their learning method was capable of producing the same classification accuracy (see Table 1) for APV, IDV, RTV and ATV, while for the remaining inhibitors it achieved better results.