Showing papers on "Interpretability published in 2006"

Unbiased Recursive Partitioning: A Conditional Inference Framework

Abstract: Recursive binary partitioning is a popular tool for regression analysis. Two fundamental problems of exhaustive search procedures usually applied to fit such models have been known for a long time: overfitting and a selection bias towards covariates with many possible splits or missing values. While pruning procedures are able to solve the overfitting problem, the variable selection bias still seriously affects the interpretability of tree-structured regression models. For some special cases unbiased procedures have been suggested, however lacking a common theoretical foundation. We propose a unified framework for recursive partitioning which embeds tree-structured regression models into a well defined theory of conditional inference procedures. Stopping criteria based on multiple test procedures are implemented and it is shown that the predictive performance of the resulting trees is as good as the performance of established exhaustive search procedures. It turns out that the partitions and therefore the...

Systematic review of the psychometric properties, interpretability and feasibility of self-report pain intensity measures for use in clinical trials in children and adolescents.

Abstract: The aim of this study was to systematically review the psychometric properties, interpretability and feasibility of self-report pain intensity measures for children and adolescents for use in clinical trials evaluating pain treatments. Databases were searched for self-report measures of single-item ratings of pain intensity for children aged 3-18 years. A total of 34 single-item self-report measures were found. The measures' psychometric properties, interpretability and feasibility, were evaluated independently by two investigators according to a set of psychometric criteria. Six single-item measures met the a priori criteria and were included in the final analysis. While these six scales were determined as psychometrically sound and show evidence of responsivity, they had varying degrees of interpretability and feasibility. No single scale was found to be optimal for use with all types of pain or across the developmental age span. Specific recommendations regarding the most psychometrically sound and feasible measures based on age/developmental level and type of pain are discussed. Future research is needed to strengthen the measurement of pain in clinical trials with children.

The Syntax of Valuation and the Interpretability of Features

Abstract: In this paper, we will present a particular proposal about the nature of agreement processes and the syntax of its output. Our proposal builds on current work, but departs from existing research in a number of ways. We hope to demonstrate that our proposals not only advance the overall understanding of agreement, but also contribute to a clearer and simpler view of a number of specific syntactic phenomena. At the heart of our proposal is a conception of agreement that draws on various traditions that view it as "feature sharing". We combine this conception with a proposal that valuation and interpretability of features are independent concepts. These ideas taken together allow us to revise existing analyses of a number of syntactic constructions. In particular, we will focus on the role of verbal tense morphology in specifying other properties of a sentence, and the comparable role played by wh-morphology in specifying clause type. Particular attention will be devoted to the syntax of raising constructions and to an analysis of sentential subjects that improves on earlier work of our own.

Nonsmooth nonnegative matrix factorization (nsNMF)

Abstract: We propose a novel nonnegative matrix factorization model that aims at finding localized, part-based, representations of nonnegative multivariate data items. Unlike the classical nonnegative matrix factorization (NMF) technique, this new model, denoted "nonsmooth nonnegative matrix factorization" (nsNMF), corresponds to the optimization of an unambiguous cost function designed to explicitly represent sparseness, in the form of nonsmoothness, which is controlled by a single parameter. In general, this method produces a set of basis and encoding vectors that are not only capable of representing the original data, but they also extract highly focalized patterns, which generally lend themselves to improved interpretability. The properties of this new method are illustrated with several data sets. Comparisons to previously published methods show that the new nsNMF method has some advantages in keeping faithfulness to the data in the achieving a high degree of sparseness for both the estimated basis and the encoding vectors and in better interpretability of the factors.

Interpretation of the Lempel-Ziv Complexity Measure in the Context of Biomedical Signal Analysis

Abstract: Lempel-Ziv complexity (LZ) and derived LZ algorithms have been extensively used to solve information theoretic problems such as coding and lossless data compression. In recent years, LZ has been widely used in biomedical applications to estimate the complexity of discrete-time signals. Despite its popularity as a complexity measure for biosignal analysis, the question of LZ interpretability and its relationship to other signal parameters and to other metrics has not been previously addressed. We have carried out an investigation aimed at gaining a better understanding of the LZ complexity itself, especially regarding its interpretability as a biomedical signal analysis technique. Our results indicate that LZ is particularly useful as a scalar metric to estimate the bandwidth of random processes and the harmonic variability in quasi-periodic signals

Large scale semi-supervised linear SVMs

Abstract: Large scale learning is often realistic only in a semi-supervised setting where a small set of labeled examples is available together with a large collection of unlabeled data. In many information retrieval and data mining applications, linear classifiers are strongly preferred because of their ease of implementation, interpretability and empirical performance. In this work, we present a family of semi-supervised linear support vector classifiers that are designed to handle partially-labeled sparse datasets with possibly very large number of examples and features. At their core, our algorithms employ recently developed modified finite Newton techniques. Our contributions in this paper are as follows: (a) We provide an implementation of Transductive SVM (TSVM) that is significantly more efficient and scalable than currently used dual techniques, for linear classification problems involving large, sparse datasets. (b) We propose a variant of TSVM that involves multiple switching of labels. Experimental results show that this variant provides an order of magnitude further improvement in training efficiency. (c) We present a new algorithm for semi-supervised learning based on a Deterministic Annealing (DA) approach. This algorithm alleviates the problem of local minimum in the TSVM optimization procedure while also being computationally attractive. We conduct an empirical study on several document classification tasks which confirms the value of our methods in large scale semi-supervised settings.

Machine Learning for Direct Marketing Response Models: Bayesian Networks with Evolutionary Programming

Abstract: Machine learning methods are powerful tools for data mining with large noisy databases and give researchers the opportunity to gain new insights into consumer behavior and to improve the performance of marketing operations. To model consumer responses to direct marketing, this study proposes Bayesian networks learned by evolutionary programming. Using a large direct marketing data set, we tested the endogeneity bias in the recency, frequency, monetary value (RFM) variables using the control function approach; compared the results of Bayesian networks with those of neural networks, classification and regression tree (CART), and latent class regression; and applied a tenfold cross-validation. The results suggest that Bayesian networks have distinct advantages over the other methods in accuracy of prediction, transparency of procedures, interpretability of results, and explanatory insight. Our findings lend strong support to Bayesian networks as a robust tool for modeling consumer response and other marketing problems and for assisting management decision making.

Learning interpretable SVMs for biological sequence classification.

Abstract: Support Vector Machines (SVMs) – using a variety of string kernels – have been successfully applied to biological sequence classification problems. While SVMs achieve high classification accuracy they lack interpretability. In many applications, it does not suffice that an algorithm just detects a biological signal in the sequence, but it should also provide means to interpret its solution in order to gain biological insight. We propose novel and efficient algorithms for solving the so-called Support Vector Multiple Kernel Learning problem. The developed techniques can be used to understand the obtained support vector decision function in order to extract biologically relevant knowledge about the sequence analysis problem at hand. We apply the proposed methods to the task of acceptor splice site prediction and to the problem of recognizing alternatively spliced exons. Our algorithms compute sparse weightings of substring locations, highlighting which parts of the sequence are important for discrimination. The proposed method is able to deal with thousands of examples while combining hundreds of kernels within reasonable time, and reliably identifies a few statistically significant positions.

On the combination of evolutionary algorithms and stratified strategies for training set selection in data mining

Abstract: In this paper, we present a new approach for training set selection in large size data sets. The algorithm consists on the combination of stratification and evolutionary algorithms. The stratification reduces the size of domain where the selection is applied while the evolutionary method selects the most representative instances. The performance of the proposal is compared with seven non-evolutionary algorithms, in stratified execution. The analysis follows two evaluating approaches: balance between reduction and accuracy of the subsets selected, and balance between interpretability and accuracy of the representation models associated to these subsets. The algorithms have been assessed on large and huge size data sets. The study shows that the stratified evolutionary instance selection consistently outperforms the non-evolutionary ones. The main advantages are: high instance reduction rates, high classification accuracy and models with high interpretability.

Hybrid learning models to get the interpretability–accuracy trade-off in fuzzy modeling

Abstract: One of the problems associated to linguistic fuzzy modeling is its lack of accuracy when modeling some complex systems. To overcome this problem, many different possibilities of improving the accuracy of linguistic fuzzy modeling have been considered in the specialized literature. We will call these approaches as basic refinement approaches. In this work, we present a short study of how these basic approaches can be combined to obtain new hybrid approaches presenting a better trade-off between interpretability and accuracy. As an example of application of these kinds of systems, we analyze seven hybrid approaches to develop accurate and still interpretable fuzzy rule-based systems, which will be tested considering two real-world problems.

Bankruptcy prediction with neural logic networks by means of grammar-guided genetic programming

Abstract: The paper demonstrates the efficient use of hybrid intelligent systems for solving the classification problem of bankruptcy. The aim of the study is to obtain classification schemes able to predict business failure. Previous attempts to form efficient classifiers for the same problem using intelligent or statistical techniques are discussed throughout the paper. The application of neural logic networks by means of genetic programming is proposed. This is an advantageous approach enabling the interpretation of the network structure through set of expert rules, which is a desirable feature for field experts. These evolutionary neural logic networks are consisted of an innovative hybrid intelligent methodology, by which evolutionary programming techniques are used for obtaining the best possible topology of a neural logic network. The genetic programming process is guided using a context-free grammar and indirect encoding of the neural logic networks into the genetic programming individuals. Indicative classification results are presented and discussed in detail in terms of both, classification accuracy and solution interpretability.

An empirical analysis of measurement equivalence with the indcol measure of individualism and collectivism: implications for valid cross‐cultural inference

Abstract: The INDCOL measure of individualism and collectivism (Singelis et al., 1995) has been used increasingly to test complex cross-cultural hypotheses. However, sample differences in translation, culture, organization, and response context might threaten the validity of cross-cultural inferences. We systematically explored the robustness of the INDCOL, for various statistical uses, in the face of those 4 threats. An analysis of measurement equivalence using multigroup mean and covariance structure analysis compared samples of INDCOL data from the United States, Singapore, and Korea. The INDCOL was robust with regard to the interpretability of correlations, whereas differences in culture and translation pose an important potential threat to the interpretability of mean-level analyses. Recommendations regarding the interpretation of the INDCOL and issues in the analysis of measurement equivalence in cross-cultural research are discussed.

From Ensemble of Fuzzy Classifiers to Single Fuzzy Rule Base Classifier

Abstract: Neuro-fuzzy systems show very good performance and the knowledge comprised within their structure is easily interpretable. To further improve their accuracy they can be combined into ensembles. In the paper we combine specially modified Mamdani neuro-fuzzy systems into an AdaBoost ensemble. The proposed modification improves the interpretability of knowledge by allowing merging the subsystems rule bases into one knowledge base. Simulations on two benchmarks shows excellent performance of the modified neuro-fuzzy systems.

A soft computing framework for software effort estimation

Abstract: Accurate software estimation such as cost estimation, quality estimation and risk analysis is a major issue in software project management. In this paper, we present a soft computing framework to tackle this challenging problem. We first use a preprocessing neuro-fuzzy inference system to handle the dependencies among contributing factors and decouple the effects of the contributing factors into individuals. Then we use a neuro-fuzzy bank to calibrate the parameters of contributing factors. In order to extend our framework into fields that lack of an appropriate algorithmic model of their own, we propose a default algorithmic model that can be replaced when a better model is available. One feature of this framework is that the architecture is inherently independent of the choice of algorithmic models or the nature of the estimation problems. By integrating neural networks, fuzzy logic and algorithmic models into one scheme, this framework has learning ability, integration capability of both expert knowledge and project data, good interpretability, and robustness to imprecise and uncertain inputs. Validation using industry project data shows that the framework produces good results when used to predict software cost.

Logic-Based Fuzzy Neurocomputing With Unineurons

Abstract: In this paper, we introduce a new category of logic neurons- unineurons that are based on the concept of uninorms. As uninorms form a certain generalization of the generic categories of fuzzy set operators such as t-norms and t-conorms, the proposed unineurons inherit their logic processing capabilities which make them flexible and logically appealing. We discuss several fundamental categories of uninorms (such as UNI_or, UNI_and, and alike). In particular, we focus on the interpretability of networks composed of unineurons leading to several categories of rules to be exploited in rule-based systems. The learning aspects of the unineurons are presented along with detailed optimization schemes. Experimental results tackle two categories of problems such as: (a) a logic approximation of fuzzy sets, and (b) a design of associations between information granules where the ensuing development schemes directly relate to the fundamentals of granular (fuzzy) modeling

Kernelizing the output of tree-based methods

Abstract: We extend tree-based methods to the prediction of structured outputs using a kernelization of the algorithm that allows one to grow trees as soon as a kernel can be defined on the output space. The resulting algorithm, called output kernel trees (OK3), generalizes classification and regression trees as well as tree-based ensemble methods in a principled way. It inherits several features of these methods such as interpretability, robustness to irrelevant variables, and input scalability. When only the Gram matrix over the outputs of the learning sample is given, it learns the output kernel as a function of inputs. We show that the proposed algorithm works well on an image reconstruction task and on a biological network inference problem.

FITSK: online local learning with generic fuzzy input Takagi-Sugeno-Kang fuzzy framework for nonlinear system estimation

Abstract: Existing Takagi-Sugeno-Kang (TSK) fuzzy models proposed in the literature attempt to optimize the global learning accuracy as well as to maintain the interpretability of the local models. Most of the proposed methods suffer from the use of offline learning algorithms to globally optimize this multi-criteria problem. Despite the ability to reach an optimal solution in terms of accuracy and interpretability, these offline methods are not suitably applicable to learning in adaptive or incremental systems. Furthermore, most of the learning methods in TSK-model are susceptible to the limitation of the curse-of-dimensionality. This paper attempts to study the criteria in the design of TSK-models. They are: 1) the interpretability of the local model; 2) the global accuracy; and 3) the system dimensionality issues. A generic framework is proposed to handle the different scenarios in this design problem. The framework is termed the generic fuzzy input Takagi-Sugeno-Kang fuzzy framework (FITSK). The FITSK framework is extensible to both the zero-order and the first-order FITSK models. A zero-order FITSK model is suitable for the learning of adaptive system, and the bias-variance of the system can be easily controlled through the degree of localization. On the other hand, a first-order FITSK model is able to achieve higher learning accuracy for nonlinear system estimation. A localized version of recursive least-squares algorithm is proposed for the parameter tuning of the first-order FITSK model. The local recursive least-squares is able to achieve a balance between interpretability and learning accuracy of a system, and possesses greater immunity to the curse-of-dimensionality. The learning algorithms for the FITSK models are online, and are readily applicable to adaptive system with fast convergence speed. Finally, a proposed guideline is discussed to handle the model selection of different FITSK models to tackle the multi-criteria design problem of applying the TSK-model. Extensive simulations were conducted using the proposed FITSK models and their learning algorithms; their performances are encouraging when benchmarked against other popular fuzzy systems.

Linear and Non-linear Credit Scoring by Combining Logistic Regression and Support Vector Machines

Abstract: The Basel II capital accord encourages banks to develop internal rating models that are financially intuitive, easily interpretable and optimally predictive for default. Standard linear logistic models are very easily readable but have limited model flexibility. Advanced neural network and support vector machine models (SVMs) are less straightforward to interpret but can capture more complex multivariate non-linear relations. A gradual approach that balances the interpretability and predictability requirements is applied here to rate banks. First, a linear model is estimated; it is then improved by identifying univariate non-linear ratio transformations that emphasize distressed conditions; and finally SVMs are added to capture remaining multivariate non-linear relations.

Predicting interpretability of metabolome models based on behavior, putative identity, and biological relevance of explanatory signals

Abstract: Powerful algorithms are required to deal with the dimensionality of metabolomics data. Although many achieve high classification accuracy, the models they generate have limited value unless it can be demonstrated that they are reproducible and statistically relevant to the biological problem under investigation. Random forest (RF) generates models, without any requirement for dimensionality reduction or feature selection, in which individual variables are ranked for significance and displayed in an explicit manner. In metabolome fingerprinting by mass spectrometry, each metabolite can be represented by signals at several m/z. Exploiting a prior understanding of expected biochemical differences between sample classes, we aimed to develop meaningful metrics relevant to the significance both of the overall RF model and individual, potentially explanatory, signals. Pair-wise comparison of related plant genotypes with strong phenotypic differences demonstrated that robust models are not only reproducible but also logically structured, highlighting correlated m/z derived from just a small number of explanatory metabolites reflecting the biological differences between sample classes. RF models were also generated by using groupings of samples known to be increasingly phenotypically similar. Although classification accuracy was often reasonable, we demonstrated reproducibly in both Arabidopsis and potato a performance threshold based on margin statistics beyond which such models showed little structure indicative of either generalizibility or further biological interpretability. In a multiclass problem using 25 Arabidopsis genotypes, despite the complicating effects of ecotype background and secondary metabolome perturbations common to several mutations, the ranking of metabolome signals by RF provided scope for deeper interpretability.

Satellite image classification using granular neural networks

Abstract: The increased synergy between neural networks (NN) and fuzzy sets has led to the introduction of granular neural networks (GNNs) that operate on granules of information, rather than information itself. The fact that processing is done on a conceptual rather than on a numerical level, combined with the representation of granules using linguistic terms, results in increased interpretability. This is the actual benefit, and not increased accuracy, gained by GNNs. The constraints used to implement the GNN are such that accuracy degradation should not be surprising. Having said that, it is well known that simple structured NNs tend to be less prone to over-fitting the training data set, maintaining the ability to generalize and more accurately classify previously unseen data. Standard NNs are frequently found to be accurate but difficult to explain, hence they are often associated with the black box syndrome. Because in GNNs the operation is carried out at a conceptual level, the components have unambiguous meaning, revealing how classification decisions are formed. In this paper, the interpretability of GNNs is exploited using a satellite image classification problem. We examine how land use classification using both spectral and non-spectral information is expressed in GNN terms. One further contribution of this paper is the use of specific symbolization of the network components to easily establish causality relationships.

Structured multicategory support vector machines with analysis of variance decomposition

Abstract: The support vector machine has been a popular choice of classification method for many applications in machine learning. While it often outperforms other methods in terms of classification accuracy, the implicit nature of its solution renders the support vector machine less attractive in providing insights into the relationship between covariates and classes. Use of structured kernels can remedy the drawback. Borrowing the flexible model-building idea of functional analysis of variance decomposition, we consider multicategory support vector machines with analysis of variance kernels in this paper. An additional penalty is imposed on the sum of weights of functional subspaces, which encourages a sparse representation of the solution. Incorporation of the additional penalty enhances the interpretability of a resulting classifier with often improved accuracy. The proposed method is demonstrated through simulation studies and an application to real data.

Solving Regression by Learning an Ensemble of Decision Rules

Abstract: We introduce a novel decision rule induction algorithm for solving the regression problem. There are only few approaches in which decision rules are applied to this type of prediction problems. The algorithm uses a single decision rule as a base classifier in the ensemble. Forward stagewise additive modeling is used in order to obtain the ensemble of decision rules. We consider two types of loss functions, the squared- and absolute-error loss, that are commonly used in regression problems. The minimization of empirical risk based on these loss functions is performed by two optimization techniques, the gradient boosting and the least angle technique. The main advantage of decision rules is their simplicity and good interpretability. The prediction model in the form of an ensemble of decision rules is powerful, which is shown by results of the experiment presented in the paper.

Coordination number prediction using learning classifier systems: performance and interpretability

Abstract: The prediction of the coordination number (CN) of an amino acid in a protein structure has recently received renewed attention. In a recent paper, Kinjo et al. proposed a real-valued definition of CN and a criterion to map it onto a finite set of classes, in order to predict it using classification approaches. The literature reports several kinds of input information used for CN prediction. The aim of this paper is to assess the performance of a state-of-the-art learning method, Learning Classifier Systems (LCS) on this CN definition, with various degrees of precision, based on several combinations of input attributes. Moreover, we will compare the LCS performance to other well-known learning techniques. Our experiments are also intended to determinethe minimum set of input information needed to achieve good predictive performance, so as to generate competent yet simple and interpretable classification rules. Thus, the generated predictors (rule sets) are analyzed for their interpretability.

Model selection in nonparametric hazard regression

Abstract: We propose a novel model selection method for a nonparametric extension of the Cox proportional hazard model, in the framework of smoothing splines ANOVA models. The method automates the model building and model selection processes simultaneously by penalizing the reproducing kernel Hilbert space norms. On the basis of a reformulation of the penalized partial likelihood, we propose an efficient algorithm to compute the estimate. The solution demonstrates great flexibility and easy interpretability in modeling relative risk functions for censored data. Adaptive choice of the smoothing parameter is discussed. Both simulations and a real example suggest that our proposal is a useful tool for multivariate function estimation and model selection in survival analysis.

ITER: an algorithm for predictive regression rule extraction

Abstract: Various benchmarking studies have shown that artificial neural networks and support vector machines have a superior performance when compared to more traditional machine learning techniques. The main resistance against these newer techniques is based on their lack of interpretability: it is difficult for the human analyst to understand the motivation behind these models’ decisions. Various rule extraction techniques have been proposed to overcome this opacity restriction. However, most of these extraction techniques are devised for classification and only few algorithms can deal with regression problems. In this paper, we present ITER, a new algorithm for pedagogical regression rule extraction. Based on a trained ‘black box’ model, ITER is able to extract human-understandable regression rules. Experiments show that the extracted model performs well in comparison with CART regression trees and various other techniques.

Speaker Verification with Confidence and Reliability Measures

Abstract: In pattern recognition, the need to quantify the quality of a classifier's output has gained importance in the past years. Speaker verification is no exception. This paper presents a probabilistic reliability framework incorporating signal-domain information into the confidence estimation and contrasts this method with classical approaches to estimating the confidence in a given speaker verification classifier output. We show that the method proposed can deal with adverse acoustic conditions for a wide range of signal-to-noise ratios, does not depend on a Gaussian assumption for impostor and client score distributions, and presents benefits in terms of scalability and interpretability of the measure. We contrast reliability and confidence approaches, and evaluate performance on a degraded version of the 295-users XM2VTS database.

Introducing the Consensus Modeling Concept in Genetic Algorithms: Application to Interpretable Discriminant Analysis

Abstract: An evolutionary statistical learning method was applied to classify drugs according to their biological target and also to discriminate between a compilation of oral and nonoral drugs. The emphasis was placed not only on how well the models predict but also on their interpretability. In an enhancement to previous studies, the consistency of the model weights over several runs of the genetic algorithm was considered with the goal of producing comprehensible models. Via this approach, the descriptors and their ranges that contribute most to class discrimination were identified. Selecting a bin step size that enables the average descriptor properties of the class being trained to be captured improves the interpretability and discriminatory power of a model. The performance, consistency, and robustness of such models were further enhanced by using two novel approaches that reduce the variability between individual solutions: consensus and splice modeling. Finally, the ability of the genetic algorithm to discriminate between activity classes was compared with a similarity searching method, while naive Bayes classifiers and support vector machines were applied in discriminating the oral and nonoral drugs.

Adaptive Fuzzy Association Rule mining for effective decision support in biomedical applications

Abstract: Due to complexity of biomedical classification problems, it is impossible to build a perfect classifier with 100% prediction accuracy. Hence a more realistic target is to build an effective Decision Support System (DSS). Here 'effective' means that a DSS should not only predict unseen samples accurately, but also work in a human-understandable way. In this paper, we propose a novel adaptive Fuzzy Association Rules (FARs) mining algorithm, named FARM-DS, to build such a DSS for binary classification problems in the biomedical domain. In the training phase, four steps are executed to mine FARs, which are thereafter used to predict unseen samples in the testing phase. The new FARM-DS algorithm is evaluated on two publicly available medical datasets. The experimental results show that FARM-DS is competitive in terms of prediction accuracy. More importantly, the mined FARs provide strong decision support on disease diagnoses due to their easy interpretability.