Machine Learning is the study of methods for programming computers to learn. Computers are applied to a wide range of tasks, and for most of these it is relatively easy for programmers to design and implement the necessary software. However, there are many tasks for which this is difficult or impossible. These can be divided into four general categories. First, there are problems for which there exist no human experts. For example, in modern automated manufacturing facilities, there is a need to predict machine failures before they occur by analyzing sensor readings. Because the machines are new, there are no human experts who can be interviewed by a programmer to provide the knowledge necessary to build a computer system. A machine learning system can study recorded data and subsequent machine failures and learn prediction rules. Second, there are problems where human experts exist, but where they are unable to explain their expertise. This is the case in many perceptual tasks, such as speech recognition, hand-writing recognition, and natural language understanding. Virtually all humans exhibit expert-level abilities on these tasks, but none of them can describe the detailed steps that they follow as they perform them. Fortunately, humans can provide machines with examples of the inputs and correct outputs for these tasks, so machine learning algorithms can learn to map the inputs to the outputs. Third, there are problems where phenomena are changing rapidly. In finance, for example, people would like to predict the future behavior of the stock market, of consumer purchases, or of exchange rates. These behaviors change frequently, so that even if a programmer could construct a good predictive computer program, it would need to be rewritten frequently. A learning program can relieve the programmer of this burden by constantly modifying and tuning a set of learned prediction rules. Fourth, there are applications that need to be customized for each computer user separately. Consider, for example, a program to filter unwanted electronic mail messages. Different users will need different filters. It is unreasonable to expect each user to program his or her own rules, and it is infeasible to provide every user with a software engineer to keep the rules up-to-date. A machine learning system can learn which mail messages the user rejects and maintain the filtering rules automatically. Machine learning addresses many of the same research questions as the fields of statistics, data mining, and psychology, but with differences of emphasis. Statistics focuses on understanding the phenomena that have generated the data, often with the goal of testing different hypotheses about those phenomena. Data mining seeks to find patterns in the data that are understandable by people. Psychological studies of human learning aspire to understand the mechanisms underlying the various learning behaviors exhibited by people (concept learning, skill acquisition, strategy change, etc.).

Machine learning

An overview of the self-organizing map algorithm, on which the papers in this issue are based, is presented in this article.

The Self-Organizing Map

http://bcpw.bg.pw.edu.pl/Content/1968/Rudiments.pdf

Rudiments of rough sets

8. Subset Selection in Regression (Monographs on Statistics and Applied Probability, no. 40). By A. J. Miller. ISBN 0 412 35380 6. Chapman and Hall, London, 1990. 240 pp. £25.00.

Subset Selection in Regression

Rough sets and Boolean reasoning

Vijay V. Raghavan Hayri Sever The Center for Advanced Computer Studies The Department of Computer Science University of Southwestern Louisiana University of Southwestern Louisiana Lafayette, LA 70504, USA Lafayette, LA 70504, USA e-mail: raghavan@cacs.usl. edu Information Retrieval (IR) systems exploit user feedback by generating an optimal query with respect to a particular information need. Since obtaining an optimal query is an expensive process, the need for mechanisms to save and reuse past optimal queries for future queries is obvions. In this article, we propose the use of a query base, a set of persistent past optimal queries, and investigate similarity measures between queries. The query base can be used either to answer user queries or to formulate optimal queries. We justify the former case analytically and the latter case by experiment.

On the reuse of past optimal queries

Data mining is an interdisciplinary research area spanning several disciplines such as database systems, machine learning, intelligent information systems, statistics, and expert systems. Data mining has evolved into an important and active area of research because of theoretical challenges and practical applications associated with the problem of discovering (or extracting) interesting and previously unknown knowledge from very large real-world databases. Many aspects of data mining have been investigated in several related fields. But the problem is unique enough that there is a great need to extend these studies to include the nature of the contents of the real-world databases. In this chapter, we discuss the theory and foundational issues in data mining, describe data mining methods and algorithms, and review data mining applications. Since a major focus of this book is on rough sets and its applications to database mining, one full section is devoted to summarizing the state of rough sets as related to data mining of real-world databases. More importantly, we provide evidence showing that the theory of rough sets constitutes a sound basis for data mining applications.

Data Mining: Trends in Research and Development

In this article, we develop and analyze four algorithms patterns from large databases. As described in Fayyad for feature selection in the context of rough set method- ( 1996 ) and Simoudis ( 1996 ) , this process is typically ology. The initial state and the feasibility criterion of all made up of selection and sampling, preprocessing and these algorithms are the same. That is, they start with a cleaning, transformation and reduction, data mining, and given feature set and progressively remove features, evaluation steps. The first step in the data-mining process while controlling the amount of degradation in classification quality. These algorithms, however, differ in the is to select a target data set from a database ( or a data heuristics used for pruning the search space of features. warehouse ) and to possibly sample the target data. The Our experimental results confirm the expected relation- preprocessing and data cleaning step handles noise and ship between the time complexity of these algorithms unknown values, as well as accounting for missing data and the classification accuracy of the resulting upper fields, time sequence information, and so forth. The data classifiers. Our experiments demonstrate that a u-reduct of a given feature set can be found efficiently. Although reduction and transformation step involves finding relewe have adopted upper classifiers in our investigations, vant features depending on the goal of the task and certain the algorithms presented can, however, be used with transformations on the data such as converting one type any method of deriving a classifier, where the quality of of data to another ( e.g., changing nominal values into classification is a monotonically decreasing function of the size of the feature set. We compare the performance numeric ones, discretizing continuous values ) , and / or deof upper classifiers with those of lower classifiers. We fining new attributes. In the mining step, the user may find that upper classifiers perform better than lower apply one or more knowledge discovery techniques on classifiers for a duodenal ulcer data set. This should be the transformed data to extract valuable patterns. Finally, generally true when there is a small number of elements the evaluation step involves interpreting the result ( or in the boundary region. An upper classifier has some important features that make it suitable for data mining discovered pattern ) with respect to the goal / task at hand. applications. In particular, we have shown that the upper Note that the data-mining process is not linear and inclassifiers can be summarized at a desired level of ab- volves a variety of feedback loops, because any one step straction by using extended decision tables. We also can result in changes in preceding or succeeding steps. point out that an upper classifier results in an inconsistent decision algorithm, which can be interpreted deter- Furthermore, the nature of a large, real-world data set, ministically or non-deterministically to obtain a consis- which may contain noisy, incomplete, dynamic, reduntent decision algorithm. dant, spare, and missing values, certainly requires that existing techniques and approaches be extended to cope with such problems ( Deogun, Raghavan, Sarkar, & Sever,

Feature selection and effective classifiers

In this paper, we investigate enhancements to an upper classifier - a decision algorithm generated by an upper classification method, which is one of the classification methods in rough set theory Specifically, we consider two enhancements First, we present a stepwise backward feature selection algorithm to preprocess a given set of features This is important because rough classification methods are incapable of removing superfluous features We prove that the stepwise backward selection algorithm finds a small subset of relevant features that are ideally sufficient and necessary to define target concepts with respect to a given threshold This threshold value indicates an acceptable degradation in the quality of an upper classifier Second, to make an upper classifier adaptive, we associate it with some kind of frequency information, which we call incremental information An extended decision table is used to represent an adaptive upper classifier It is also used for interpreting an upper classifier either deterministically or nondeterministically

/pdf/exploiting-upper-approximation-in-the-rough-set-methodology-1pcszlslsk.pdf

Exploiting upper approximation in the rough set methodology

We study the feature selection problem and develop and analyze four algorithms for feature selection in the context of rough set methodology The initial state and the feasibility criterion of all these algorithms are the same, that is, they start from a given feature set and progressively remove features, while controlling the amount of degradation in classification quality, but differ in the heuristic used for pruning the search space of features Our experimental results confirm the analytical results on the complexity of algorithms as well as on controlled degradation of upper classification The algorithms presented can be used with any methods of deriving a classifier where the quality of classification is a monotonically decreasing function while feature set is reduced, though we have adopted the upper classifier in our study The upper classifier has some important features that makes it suitable for database mining applications In particular, we have shown that the upper classifier can be summarized at a desired level of abstraction by using extended decision tables We also point out that an inconsistent decision algorithm can be interpreted as if it were a consistent decision algorithm

Hayri Sever

Papers

On the reuse of past optimal queries

Data Mining: Trends in Research and Development

Feature selection and effective classifiers

Exploiting upper approximation in the rough set methodology

A comparison of feature selection algorithms in the context of rough classifiers