Machine Learning is the study of methods for programming computers to learn. Computers are applied to a wide range of tasks, and for most of these it is relatively easy for programmers to design and implement the necessary software. However, there are many tasks for which this is difficult or impossible. These can be divided into four general categories. First, there are problems for which there exist no human experts. For example, in modern automated manufacturing facilities, there is a need to predict machine failures before they occur by analyzing sensor readings. Because the machines are new, there are no human experts who can be interviewed by a programmer to provide the knowledge necessary to build a computer system. A machine learning system can study recorded data and subsequent machine failures and learn prediction rules. Second, there are problems where human experts exist, but where they are unable to explain their expertise. This is the case in many perceptual tasks, such as speech recognition, hand-writing recognition, and natural language understanding. Virtually all humans exhibit expert-level abilities on these tasks, but none of them can describe the detailed steps that they follow as they perform them. Fortunately, humans can provide machines with examples of the inputs and correct outputs for these tasks, so machine learning algorithms can learn to map the inputs to the outputs. Third, there are problems where phenomena are changing rapidly. In finance, for example, people would like to predict the future behavior of the stock market, of consumer purchases, or of exchange rates. These behaviors change frequently, so that even if a programmer could construct a good predictive computer program, it would need to be rewritten frequently. A learning program can relieve the programmer of this burden by constantly modifying and tuning a set of learned prediction rules. Fourth, there are applications that need to be customized for each computer user separately. Consider, for example, a program to filter unwanted electronic mail messages. Different users will need different filters. It is unreasonable to expect each user to program his or her own rules, and it is infeasible to provide every user with a software engineer to keep the rules up-to-date. A machine learning system can learn which mail messages the user rejects and maintain the filtering rules automatically. Machine learning addresses many of the same research questions as the fields of statistics, data mining, and psychology, but with differences of emphasis. Statistics focuses on understanding the phenomena that have generated the data, often with the goal of testing different hypotheses about those phenomena. Data mining seeks to find patterns in the data that are understandable by people. Psychological studies of human learning aspire to understand the mechanisms underlying the various learning behaviors exhibited by people (concept learning, skill acquisition, strategy change, etc.).

Machine learning

ACM-SIAM Symposium on Discrete Algorithms

Artificial intelligence (AI) and machine learning (ML) have recently been radically improved and are now being employed in almost every application domain to develop automated or semi-automated systems. To facilitate greater human acceptability of these systems, explainable artificial intelligence (XAI) has experienced significant growth over the last couple of years with the development of highly accurate models but with a paucity of explainability and interpretability. The literature shows evidence from numerous studies on the philosophy and methodologies of XAI. Nonetheless, there is an evident scarcity of secondary studies in connection with the application domains and tasks, let alone review studies following prescribed guidelines, that can enable researchers’ understanding of the current trends in XAI, which could lead to future research for domain- and application-specific method development. Therefore, this paper presents a systematic literature review (SLR) on the recent developments of XAI methods and evaluation metrics concerning different application domains and tasks. This study considers 137 articles published in recent years and identified through the prominent bibliographic databases. This systematic synthesis of research articles resulted in several analytical findings: XAI methods are mostly developed for safety-critical domains worldwide, deep learning and ensemble models are being exploited more than other types of AI/ML models, visual explanations are more acceptable to end-users and robust evaluation metrics are being developed to assess the quality of explanations. Research studies have been performed on the addition of explanations to widely used AI/ML models for expert users. However, more attention is required to generate explanations for general users from sensitive domains such as finance and the judicial system.

/pdf/a-systematic-review-of-explainable-artificial-intelligence-1fw11vjz.pdf

A Systematic Review of Explainable Artificial Intelligence in Terms of Different Application Domains and Tasks

This work explores student’s academic performance using decision tree algorithm having parameters like Student’s Academic Information and Students Activity. We collected records of 22 students from Spring 2017 semester, studying in undergraduate level from Oman’s private Higher Education Institution. Proposed work utilizes Electronic Commerce Technologies module since it is a core module offered in every computing specialization. Furthermore, WEKA data mining tool is used to evaluate the decision tree algorithm for discovery of student’s performance along with Moodle access time. Simulation results demonstrate that Random Forest Tree algorithm showed better accuracy than comparative decision tree algorithms. Hence, shows good agreement for the training set provided. Therefore, the proposed work aid in improving student’s grades in the module. Helping stakeholders to analyze and evaluate the module delivery and results. Early detection and solution can be made both at the institutional level and module level.

Student Academic Performance Prediction by using Decision Tree Algorithm

Combinatorial testing has been shown to be a very effective strategy for software testing. After a failure is detected, the next task is to identify one or more faulty statements in the source code that have caused the failure. In this paper, we present a fault localization approach, called BEN, which produces a ranking of statements in terms of their likelihood of being faulty by leveraging the result of combinatorial testing. BEN consists of two major phases. In the first phase, BEN identifies a combination that is very likely to be failure-inducing. A combination is failure-inducing if it causes any test in which it appears to fail. In the second phase, BEN takes as input a failure-inducing combination identified in the first phase and produces a ranking of statements in terms of their likelihood to be faulty. We conducted an experiment in which our approach was applied to the Siemens suite and four real-world programs, flex , grep, gzip and sed , from Software Infrastructure Repository (SIR). The experimental results show that our approach can effectively and efficiently localize the faulty statements in these programs.

A Combinatorial Testing-Based Approach to Fault Localization

We present a combinatorial testing-based fault localization tool called BEN. BEN takes as input three types of information, including the subject program, the source code, an input parameter model, and a combinatorial test set created based on the input parameter model. It is assumed that the combinatorial test set has already been executed, and thus the execution status of each test is known. The output of BEN is a ranking of statements in terms of their likelihood to be faulty. In the fault localization process, a small number of additional tests are generated by BEN and need to be executed by the user. In this paper, we present the major user scenarios and the highlevel design of BEN. BEN is implemented in Java and provides a graphical user interface that provides friendly access to the tool.

BEN: A combinatorial testing-based fault localization tool

Data mining algorithms are used to analyze and discover useful information from data. This paper presents an experiment that applies Combinatorial Testing (CT) to five data mining algorithms implemented in an open-source data mining software called WEKA. For each algorithm, we first run the algorithm with 51 datasets to study the impact different datasets have on the test coverage. We select one dataset that achieves the highest branch coverage. Next we construct positive and negative combinatorial test sets of configuration options and execute each test set with the selected dataset. Test effectiveness is measured using branch and mutation coverage. Our results suggest that when testing data mining algorithms: (1) larger datasets do not necessarily achieve higher coverage than smaller datasets, (2) test coverage increases progressively slower as test strength increases, and (3) branch coverage correlates well with mutation coverage.

Applying Combinatorial Testing to Data Mining Algorithms

Debugging or fault localization is one of the most challenging tasks during software development. Many tools have been developed to reduce the amount of effort and time software developers have to spend on fault localization. In this paper, we evaluate the effectiveness of a fault localization tool called BEN in localizing different types of software fault. Assuming that combinatorial testing has been performed on the subject program, BEN leverages the result obtained from combinatorial testing to perform fault localization. Our evaluation focuses on impact of three properties of software fault on the effectiveness of BEN. The three properties include accessibility, input value sensitivity and control flow sensitivity. A random test set-based approach is used to measure the three properties. The experimental results suggest that BEN is more effective, respectively, in localizing faults of lower accessibility, input value-insensitive faults or control flow-insensitive faults than localizing faults of higher accessibility, input value-sensitive or control flow-sensitive faults in the subject programs. The insights obtained from our evaluation can be applied to other fault localization tools that are similar to BEN, and can be used to identify opportunities for further research on combinatorial testing-based fault localization.

/pdf/evaluating-the-effectiveness-of-ben-in-localizing-different-44ijhmqu1q.pdf

Evaluating the Effectiveness of BEN in Localizing Different Types of Software Fault

Machine Learning (ML) models, a core component to artificial intelligence systems, often come as a black box to the user, leading to the problem of interpretability. Explainable Artificial Intelligence (XAI) is key to providing confidence and trustworthiness for machine learning-based software systems. We observe a fundamental connection between XAI and software fault localization. In this paper, we present an approach that uses BEN, a combinatorial testing-based software fault localization approach, to produce explanations for decisions made by ML models.

A Combinatorial Approach to Explaining Image Classifiers

Many machine learning algorithms examine large amounts of data to discover insights from hidden patterns. Testing these algorithms can be expensive and time-consuming. There is a need to speed up the testing process, especially in an agile development process, where testing is frequently performed. One approach is to replace big datasets with smaller datasets produced by random sampling. In this paper, we report a set of experiments that are designed to evaluate the effectiveness of using reduced datasets produced by random sampling for testing machine learning algorithms. In our experiments, we use as subject programs four supervised learning algorithms from the Waikato Environment for Knowledge Analysis (WEKA). We identify five datasets from Kaggle.com to run with the four learning algorithms. For each dataset, we generate reduced datasets of different sizes using two random sampling strategies, i.e., pure random and stratified random sampling. We execute our subject programs with the original and the reduced datasets, and measure test effectiveness using branch and mutation coverage. Our results indicate that in most cases, reduced datasets of even very small sizes can achieve the same or similar coverage achieved by the original dataset. Furthermore, our results indicate that reduced datasets produced by the two sample strategies do not differ significantly, and branch coverage correlates with mutation coverage.

Jaganmohan Chandrasekaran

Papers

BEN: A combinatorial testing-based fault localization tool

Applying Combinatorial Testing to Data Mining Algorithms

Evaluating the Effectiveness of BEN in Localizing Different Types of Software Fault

A Combinatorial Approach to Explaining Image Classifiers

Effectiveness of dataset reduction in testing machine learning algorithms