Variable and feature selection have become the focus of much research in areas of application for which datasets with tens or hundreds of thousands of variables are available. These areas include text processing of internet documents, gene expression array analysis, and combinatorial chemistry. The objective of variable selection is three-fold: improving the prediction performance of the predictors, providing faster and more cost-effective predictors, and providing a better understanding of the underlying process that generated the data. The contributions of this special issue cover a wide range of aspects of such problems: providing a better definition of the objective function, feature construction, feature ranking, multivariate feature selection, efficient search methods, and feature validity assessment methods.

/pdf/an-introduction-to-variable-and-feature-selection-22eu49r0zd.pdf

An introduction to variable and feature selection

Machine Learning is the study of methods for programming computers to learn. Computers are applied to a wide range of tasks, and for most of these it is relatively easy for programmers to design and implement the necessary software. However, there are many tasks for which this is difficult or impossible. These can be divided into four general categories. First, there are problems for which there exist no human experts. For example, in modern automated manufacturing facilities, there is a need to predict machine failures before they occur by analyzing sensor readings. Because the machines are new, there are no human experts who can be interviewed by a programmer to provide the knowledge necessary to build a computer system. A machine learning system can study recorded data and subsequent machine failures and learn prediction rules. Second, there are problems where human experts exist, but where they are unable to explain their expertise. This is the case in many perceptual tasks, such as speech recognition, hand-writing recognition, and natural language understanding. Virtually all humans exhibit expert-level abilities on these tasks, but none of them can describe the detailed steps that they follow as they perform them. Fortunately, humans can provide machines with examples of the inputs and correct outputs for these tasks, so machine learning algorithms can learn to map the inputs to the outputs. Third, there are problems where phenomena are changing rapidly. In finance, for example, people would like to predict the future behavior of the stock market, of consumer purchases, or of exchange rates. These behaviors change frequently, so that even if a programmer could construct a good predictive computer program, it would need to be rewritten frequently. A learning program can relieve the programmer of this burden by constantly modifying and tuning a set of learned prediction rules. Fourth, there are applications that need to be customized for each computer user separately. Consider, for example, a program to filter unwanted electronic mail messages. Different users will need different filters. It is unreasonable to expect each user to program his or her own rules, and it is infeasible to provide every user with a software engineer to keep the rules up-to-date. A machine learning system can learn which mail messages the user rejects and maintain the filtering rules automatically. Machine learning addresses many of the same research questions as the fields of statistics, data mining, and psychology, but with differences of emphasis. Statistics focuses on understanding the phenomena that have generated the data, often with the goal of testing different hypotheses about those phenomena. Data mining seeks to find patterns in the data that are understandable by people. Psychological studies of human learning aspire to understand the mechanisms underlying the various learning behaviors exhibited by people (concept learning, skill acquisition, strategy change, etc.).

Machine learning

http://romisatriawahono.net/lecture/rm/survey/machine%20learning/Chandrashekar%20-%20Feature%20Selection%20Methods%20-%202014.pdf

A survey on feature selection methods

General Topology-I

1. Introduction to text mining 2. Core text mining operations 3. Text mining preprocessing techniques 4. Categorization 5. Clustering 6. Information extraction 7. Probabilistic models for Information extraction 8. Preprocessing applications using probabilistic and hybrid approaches 9. Presentation-layer considerations for browsing and query refinement 10. Visualization approaches 11. Link analysis 12. Text mining applications Appendix Bibliography.

/pdf/the-text-mining-handbook-advanced-approaches-in-analyzing-ge55wa29fm.pdf

The Text Mining Handbook: Advanced Approaches in Analyzing Unstructured Data

We study an approach to text categorization that combines distributional clustering of words and a Support Vector Machine (SVM) classifier. This word-cluster representation is computed using the recently introduced Information Bottleneck method, which generates a compact and efficient representation of documents. When combined with the classification power of the SVM, this method yields high performance in text categorization. This novel combination of SVM with word-cluster representation is compared with SVM-based categorization using the simpler bag-of-words (BOW) representation. The comparison is performed over three known datasets. On one of these datasets (the 20 Newsgroups) the method based on word clusters significantly outperforms the word-based representation in terms of categorization accuracy or representation efficiency. On the two other sets (Reuters-21578 and WebKB) the word-based representation slightly outperforms the word-cluster representation. We investigate the potential reasons for this behavior and relate it to structural differences between the datasets.

/pdf/distributional-word-clusters-vs-words-for-text-4o5dxa41cn.pdf

Distributional word clusters vs. words for text categorization

En prenant en compte le comportement exceptionnel de la portee des SN indefinis et les complications ajoutees par les effets de distributivite des indefinis pluriels, l'A propose une revision des approches standard de la syntaxe et de la semantique des indefinis En suivant l'article de Reinhart Quantifier scope (1997), il explique l'unique comportement de la portee des indefinis pluriels et singuliers par une semantique de la fonction du choix, et non par quelque operation de changement de portee Mais, a la difference de Reinhart qui s'occupe aussi de la quantification standard, il considere que la quantification par les fonctions de choix est le mecanisme uniforme pour l'interpretation des indefinis Il propose en outre une definition formelle linguistiquement adequate et exploitable dans un systeme compositionnel

Choice functions and the scopal semantics of indefinites

This paper studies a distinction that was proposed in previous works between total and partial adjectives. In pairs of adjectives such as safe–dangerous, clean–dirty and healthy–sick, the first (“total”) adjective describes lack of danger, dirt, malady, etc., while the second (“partial”) adjective describes the existence of such properties. It is shown that the semantics of adjective phrases with modifiers such as almost, slightly, and completely is sensitive to whether the adjective is total or partial. The interpretation of such modified constructions is accounted for using a novel scale structure for total and partial adjectives. It is proposed that the standard value of a total adjective is always fixed as the lower bound of the corresponding partial adjective. By contrast, the standard value of partial adjectives can take any point on the partial scale. The effects of this theoretical distinction on the behavior of modified constructions are studied in detail, and their ramifications for the semantic theory of adjectives are discussed. Some other phenomena are surveyed that show evidence for total and partial adjectival constructions with various comparatives and exceptive phrases.

Total adjectives vs. partial adjectives: Scale structure and higher-order modifiers

This paper introduces a compositional semantics of locative prepositional phrases which is based on a vector space ontology. Model-theoretic properties of prepositions like monotonicity and conservativity are defined in this system in a straightforward way. These notions are shown to describe central inferences with spatial expressions and to account for the grammaticality of preposition modification. Model-theoretic constraints on the set of possible prepositions in natural language are specified, similar to the semantic universals of Generalized Quantifier Theory.

Vector Space Semantics: A Model-Theoretic Analysis of Locative Prepositions

We describe a text categorization approach that is based on a combination of feature distributional clusters with a support vector machine (SVM) classifier. Our feature selection approach employs distributional clustering of words via the recently introducedinformation bottleneck method, which generates a more efficientword-clusterrepresentation of documents. Combined with the classification power of an SVM, this method yields high performance text categorization that can outperform other recent methods in terms of categorization accuracy and representation efficiency. Comparing the accuracy of our method with other techniques, we observe significant dependency of the results on the data set. We discuss the potential reasons for this dependency.

/pdf/on-feature-distributional-clustering-for-text-categorization-1anrrbavvu.pdf

Yoad Winter

Papers

Distributional word clusters vs. words for text categorization

Choice functions and the scopal semantics of indefinites

Total adjectives vs. partial adjectives: Scale structure and higher-order modifiers

Vector Space Semantics: A Model-Theoretic Analysis of Locative Prepositions

On feature distributional clustering for text categorization