Data Mining: Practical Machine Learning Tools and Techniques offers a thorough grounding in machine learning concepts as well as practical advice on applying machine learning tools and techniques in real-world data mining situations. This highly anticipated third edition of the most acclaimed work on data mining and machine learning will teach you everything you need to know about preparing inputs, interpreting outputs, evaluating results, and the algorithmic methods at the heart of successful data mining. Thorough updates reflect the technical changes and modernizations that have taken place in the field since the last edition, including new material on Data Transformations, Ensemble Learning, Massive Data Sets, Multi-instance Learning, plus a new version of the popular Weka machine learning software developed by the authors. Witten, Frank, and Hall include both tried-and-true techniques of today as well as methods at the leading edge of contemporary research. *Provides a thorough grounding in machine learning concepts as well as practical advice on applying the tools and techniques to your data mining projects *Offers concrete tips and techniques for performance improvement that work by transforming the input or output in machine learning methods *Includes downloadable Weka software toolkit, a collection of machine learning algorithms for data mining tasks-in an updated, interactive interface. Algorithms in toolkit cover: data pre-processing, classification, regression, clustering, association rules, visualization

Data Mining: Practical Machine Learning Tools and Techniques

Variable and feature selection have become the focus of much research in areas of application for which datasets with tens or hundreds of thousands of variables are available. These areas include text processing of internet documents, gene expression array analysis, and combinatorial chemistry. The objective of variable selection is three-fold: improving the prediction performance of the predictors, providing faster and more cost-effective predictors, and providing a better understanding of the underlying process that generated the data. The contributions of this special issue cover a wide range of aspects of such problems: providing a better definition of the objective function, feature construction, feature ranking, multivariate feature selection, efficient search methods, and feature validity assessment methods.

/pdf/an-introduction-to-variable-and-feature-selection-22eu49r0zd.pdf

An introduction to variable and feature selection

https://ai.stanford.edu/~ronnyk/wrappersPrint.pdf

Wrappers for feature subset selection

General Strategies.- Regression Models.- Classification Models.- Other Considerations.- Appendix.- References.- Indices.

Applied Predictive Modeling

A central problem in machine learning is identifying a representative set of features from which to construct a classification model for a particular task. This thesis addresses the problem of feature selection for machine learning through a correlation based approach. The central hypothesis is that good feature sets contain features that are highly correlated with the class, yet uncorrelated with each other. A feature evaluation formula, based on ideas from test theory, provides an operational definition of this hypothesis. CFS (Correlation based Feature Selection) is an algorithm that couples this evaluation formula with an appropriate correlation measure and a heuristic search strategy. CFS was evaluated by experiments on artificial and natural datasets. Three machine learning algorithms were used: C4.5 (a decision tree learner), IB1 (an instance based learner), and naive Bayes. Experiments on artificial datasets showed that CFS quickly identifies and screens irrelevant, redundant, and noisy features, and identifies relevant features as long as their relevance does not strongly depend on other features. On natural domains, CFS typically eliminated well over half the features. In most cases, classification accuracy using the reduced feature set equaled or bettered accuracy using the complete feature set. Feature selection degraded machine learning performance in cases where some features were eliminated which were highly predictive of very small areas of the instance space. Further experiments compared CFS with a wrapper—a well known approach to feature selection that employs the target learning algorithm to evaluate feature sets. In many cases CFS gave comparable results to the wrapper, and in general, outperformed the wrapper on small datasets. CFS executes many times faster than the wrapper, which allows it to scale to larger datasets. Two methods of extending CFS to handle feature interaction are presented and experimentally evaluated. The first considers pairs of features and the second incorporates iii feature weights calculated by the RELIEF algorithm. Experiments on artificial domains showed that both methods were able to identify interacting features. On natural domains, the pairwise method gave more reliable results than using weights provided by RELIEF.

/pdf/correlation-based-feature-selection-for-machine-learning-578rirmvg5.pdf

Correlation-based Feature Selection for Machine Learning

In real-world concept learning problems, the representation of data often uses many features, only a few of which may be related to the target concept. In this situation, feature selection is important both to speed up learning and to improve concept quality. A new feature selection algorithm Relief uses a statistical method and avoids heuristic search. Relief requires linear time in the number of given features and the number of training instances regardless of the target concept to be learned. Although the algorithm does not necessarily find the smallest subset of features, the size tends to be small because only statistically relevant features are selected. This paper focuses on empirical test results in two artificial domains; the LED Display domain and the Parity domain with and without noise. Comparison with other feature selection algorithms shows Relief's advantages in terms of learning time and the accuracy of the learned concept, suggesting Relief's practicality.

A Practical Approach to Feature Selection

For real-world concept learning problems, feature selection is important to speed up learning and to improve concept quality We review and analyze past approaches to feature selection and note their strengths and weaknesses We then introduce and theoretically examine a new algorithm Rellef which selects relevant features using a statistical method Relief does not depend on heuristics, is accurate even if features interact, and is noise-tolerant It requires only linear time in the number of given features and the number of training instances, regardless of the target concept complexity The algorithm also has certain limitations such as nonoptimal feature set size Ways to overcome the limitations are suggested We also report the test results of comparison between Relief and other feature selection algorithms The empirical results support the theoretical analysis, suggesting a practical approach to feature selection for real-world problems

The feature selection problem: traditional methods and a new algorithm

PROBLEM TO BE SOLVED: To easily and properly comprehend contents and to provide exact information even concerning any technical inquiry by extracting a designated word and presenting data for supplementarily explaining its contents when there is a technically difficult word, etc SOLUTION: When guidance data are received from a guidance I/F device J1, a supplementary explanation extracting/presenting device G2-1 extracts the word for which a supplementary explanation is required Concretely, words registered in a supplementary explanation word dictionary G2-2 are successively read and text matching is performed to show whether such a word is contained in the guidance data or not When that word is contained, it is outputted as a word requiring the supplementary explanation Then, guidance data and supplementary exaplanation data related to the extracted word are finally presented from the guidance I/F device J1 to a user Thus, even the inquiry from an ordinary client, who does not have a through knowledge of technique, can be dealt with

Information retrieval supporting device

In this paper is presented a method of maintenance part information retrieval, applied to a multimedia database of elevator parts.In design and maintenance sections, there are needs to treat multimedia data suchas images and graphs, used for non-procedural operations, as CAD and office automation systems become disseminated. We developed a prototype system for retrieval of maintenance part information, applied to a multimedia database of elevator parts, to beused for ordering the needed information for maintenance. Conventionally, these neededpart information are searched and ordered by a maintenance engineer with cumbersome process. Namely, the engineer must select appropriate parts according both to the specifications of the elevator to be maintained, and to the description in documents foreach maintenance part.In our system, the elevator parts, which have complex hierarchical structure, are modelled by object-oriented technology. Thus, a maintenance engineer can easily retrieve and order the needed part information, guided by the computer system, utilizing

Development of object-oriented multimedia database and its application to retrieval of maintenance parts

PURPOSE:To obtain a maintenance parts retrieving device enabling also a worker not to be an expert to input a construction instruction and easily retrieve preparing parts corresonding to an equipment to be maintained by successively tracing relational parts from specified parts. CONSTITUTION:This maintenance parts retrieving device is provided with a parts structure information data base 1 for storing relational information among maintenance parts and the attribute information of parts, a parts selecting information data base 10 for storing parts selecting information for specifying parts corresponding to an equipment to be maintained, a master parts specifying information data base 20 for storing master parts specifying information, a keyboard 8 for indicating the equipment to be maintained, a master parts retrieving program 4a for retrieving structural master parts in maintenance parts corresponding to construction indicated by a construction instruction, a parts selecting program 12 for specifying parts corresponding to the equipment out of plural similar parts, and a parts retrieving program 6 for retrieving parts.

Kenji Kira

Papers

A Practical Approach to Feature Selection

The feature selection problem: traditional methods and a new algorithm

Information retrieval supporting device

Development of object-oriented multimedia database and its application to retrieval of maintenance parts

Hierarchical data retrieving device and maintenance parts retrieving device