Low-level interpretability and high-level interpretability: a unified view of data-driven interpretable fuzzy system modelling

The identification of non-linear systems using only observed finite datasets has become a mature research area over the last two decades. A class of linear-in-the-parameter models with universal approximation capabilities have been intensively studied and widely used due to the availability of many linear-learning algorithms and their inherent convergence conditions. This article presents a systematic overview of basic research on model selection approaches for linear-in-the-parameter models. One of the fundamental problems in non-linear system identification is to find the minimal model with the best model generalisation performance from observational data only. The important concepts in achieving good model generalisation used in various non-linear system-identification algorithms are first reviewed, including Bayesian parameter regularisation and models selective criteria based on the cross validation and experimental design. A significant advance in machine learning has been the development of the support vector machine as a means for identifying kernel models based on the structural risk minimisation principle. The developments on the convex optimisation-based model construction algorithms including the support vector regression algorithms are outlined. Input selection algorithms and on-line system identification algorithms are also included in this review. Finally, some industrial applications of non-linear models are discussed.

/pdf/model-selection-approaches-for-non-linear-system-2wlzndhs06.pdf

Model selection approaches for non-linear system identification: a review

The prediction accuracy and generalization ability of neural/neurofuzzy models for chaotic time series prediction highly depends on employed network model as well as learning algorithm. In this study, several neural and neurofuzzy models with different learning algorithms are examined for prediction of several benchmark chaotic systems and time series. The prediction performance of locally linear neurofuzzy models with recently developed Locally Linear Model Tree (LoLiMoT) learning algorithm is compared with that of Radial Basis Function (RBF) neural network with Orthogonal Least Squares (OLS) learning algorithm, MultiLayer Perceptron neural network with error back-propagation learning algorithm, and Adaptive Network based Fuzzy Inference System. Particularly, cross validation techniques based on the evaluation of error indices on multiple validation sets is utilized to optimize the number of neurons and to prevent over fitting in the incremental learning algorithms. To make a fair comparison between neural and neurofuzzy models, they are compared at their best structure based on their prediction accuracy, generalization, and computational complexity. The experiments are basically designed to analyze the generalization capability and accuracy of the learning techniques when dealing with limited number of training samples from deterministic chaotic time series, but the effect of noise on the performance of the techniques is also considered. Various chaotic systems and time series including Lorenz system, Mackey-Glass chaotic equation, Henon map, AE geomagnetic activity index, and sunspot numbers are examined as case studies. The obtained results indicate the superior performance of incremental learning algorithms and their respective networks, such as, OLS for RBF network and LoLiMoT for locally linear neurofuzzy model.

Predicting Chaotic Time Series Using Neural and Neurofuzzy Models: A Comparative Study

In the past couple of years, there has been increasing interest in the fusion of neural networks and fuzzy logic. Most of the existing fuzzy neural network (FNN) models have been proposed to implement different types of single-stage fuzzy reasoning mechanisms and inevitably they suffer from the dimensionality problem when dealing with complex real-world problems. To address the problem, FNN modeling based on multistage fuzzy reasoning (MSFR) is pursued here and two hierarchical network models, namely incremental type and aggregated type, are introduced. The new models called multistage FNN (MSFNN) model a hierarchical fuzzy rule set that allows the consequence of a rule passed to another as a fact through the intermediate variables. From the stipulated input-output data pairs, they can generate an appropriate fuzzy rule set through structure and parameter learning procedures proposed in this paper. In addition, we have particularly addressed the input selection problem of these two types of multistage network models and proposed two efficient methods for them. The effectiveness of the proposed MSFNN models in handling high-dimensional problems is demonstrated through various numerical simulations.

On multistage fuzzy neural network modeling

This first technical session is further developing the technological dimensions of technology transfer, with three illustrations of successful and representative industrial applications. It covers the cases of embedded autonomy in spacecraft applications, advanced avionics solutions for satellite communication and component hybridization for navigation.

Industrial applications

System identification is the task of constructing representative models of processes and has become an invaluable tool in many different areas of science and engineering. Due to the inherent complexity of many real world systems the application of traditional techniques is limited. In such instances more sophisticated (so called intelligent) modelling approaches are required. Neurofuzzy modelling is one such technique, which by integrating the attributes of fuzzy systems and neural networks is ideally suited to system identification. This attractive paradigm combines the well established learning techniques of a particular form of neural network i.e. generalised linear models with the transparent knowledge representation of fuzzy systems, thus producing models which possess the ability to learn from real world observations and whose behaviour can be described naturally as a series of linguistic humanly understandable rules. Unfortunately, the application of these systems is limited to low dimensional problems for which good quality expert knowledge and data are available. The work described in this thesis addresses this fundamental problem with neurofuzzy modelling, as a result algorithms which are less sensitive to the quality of the a priori knowledge and empirical data are developed. The true modelling capabilities of any strategy is heavily reliant on the model's structure, and hence an important (arguably the most important) task is structure identification. Also, due to the curse of dimensionality, in high dimensional problems the size of conventional neurofuzzy models gets prohibitively large. These issues are tackled by the development of automatic neurofuzzy model identification algorithms, which exploit the available expert knowledge and empirical data. To alleviate problems associated with the curse of dimensionality, aid model generalisation and enhance model transparency, parsimonious models are identified. This is achieved by the application of additive and multiplicative neurofuzzy models which exploit structural redundancies found in conventional systems. The developed construction algorithms successfully identify parsimonious models, but as a result of noisy and poorly distributed empirical data, these models can still generalise inadequately. This problem is addressed by the application of Bayesian inferencing techniques; a form of regularisation. Smooth model outputs are assumed and superfluous model parameters are controlled, sufficiently aiding model generalisation and transparency, and data interpolation and extrapolation. By exploiting the structural decomposition of the identified neurofuzzy models, an efficient local method of regularisation is developed. All the methods introduced in this thesis are illustrated on many different examples, including simulated time series, complex functional equations, and multi-dimensional dynamical systems. For many of these problems conventional neurofuzzy modelling is unsuitable, and the developed techniques have extended the range of problems to which neurofuzzy modelling can successfully be applied.

Kevin Martin Bossley

Papers

Neurofuzzy modelling approaches in system identification