Machine Learning is the study of methods for programming computers to learn. Computers are applied to a wide range of tasks, and for most of these it is relatively easy for programmers to design and implement the necessary software. However, there are many tasks for which this is difficult or impossible. These can be divided into four general categories. First, there are problems for which there exist no human experts. For example, in modern automated manufacturing facilities, there is a need to predict machine failures before they occur by analyzing sensor readings. Because the machines are new, there are no human experts who can be interviewed by a programmer to provide the knowledge necessary to build a computer system. A machine learning system can study recorded data and subsequent machine failures and learn prediction rules. Second, there are problems where human experts exist, but where they are unable to explain their expertise. This is the case in many perceptual tasks, such as speech recognition, hand-writing recognition, and natural language understanding. Virtually all humans exhibit expert-level abilities on these tasks, but none of them can describe the detailed steps that they follow as they perform them. Fortunately, humans can provide machines with examples of the inputs and correct outputs for these tasks, so machine learning algorithms can learn to map the inputs to the outputs. Third, there are problems where phenomena are changing rapidly. In finance, for example, people would like to predict the future behavior of the stock market, of consumer purchases, or of exchange rates. These behaviors change frequently, so that even if a programmer could construct a good predictive computer program, it would need to be rewritten frequently. A learning program can relieve the programmer of this burden by constantly modifying and tuning a set of learned prediction rules. Fourth, there are applications that need to be customized for each computer user separately. Consider, for example, a program to filter unwanted electronic mail messages. Different users will need different filters. It is unreasonable to expect each user to program his or her own rules, and it is infeasible to provide every user with a software engineer to keep the rules up-to-date. A machine learning system can learn which mail messages the user rejects and maintain the filtering rules automatically. Machine learning addresses many of the same research questions as the fields of statistics, data mining, and psychology, but with differences of emphasis. Statistics focuses on understanding the phenomena that have generated the data, often with the goal of testing different hypotheses about those phenomena. Data mining seeks to find patterns in the data that are understandable by people. Psychological studies of human learning aspire to understand the mechanisms underlying the various learning behaviors exhibited by people (concept learning, skill acquisition, strategy change, etc.).

Machine learning

Data Mining Practical Machine Learning Tools and Techniques

Swarm intelligence is a relatively new approach to problem solving that takes inspiration from the social behaviors of insects and of other animals In particular, ants have inspired a number of methods and techniques among which the most studied and the most successful is the general purpose optimization technique known as ant colony optimization Ant colony optimization (ACO) takes inspiration from the foraging behavior of some ant species These ants deposit pheromone on the ground in order to mark some favorable path that should be followed by other members of the colony Ant colony optimization exploits a similar mechanism for solving optimization problems From the early nineties, when the first ant colony optimization algorithm was proposed, ACO attracted the attention of increasing numbers of researchers and many successful applications are now available Moreover, a substantial corpus of theoretical results is becoming available that provides useful guidelines to researchers and practitioners in further applications of ACO The goal of this article is to introduce ant colony optimization and to survey its most notable applications

/pdf/ant-colony-optimization-4lg01ityvg.pdf

Ant Colony Optimization

Ant Colony Optimization (ACO) is a stochastic local search method that has been inspired by the pheromone trail laying and following behavior of some ant species [1]. Artificial ants in ACO essentially are randomized construction procedures that generate solutions based on (artificial) pheromone trails and heuristic information that are associated to solution components. Since the first ACO algorithm has been proposed in 1991, this algorithmic method has attracted a large number of researchers and in the meantime it has reached a significant level of maturity. In fact, ACO is now a well-established search technique for tackling a wide variety of computationally hard problems.

/pdf/ant-colony-optimization-34xgx5rdns.pdf

The introduction of ant colony optimization (ACO) and to survey its most notable applications are discussed. Ant colony optimization takes inspiration from the forging behavior of some ant species. These ants deposit Pheromone on the ground in order to mark some favorable path that should be followed by other members of the colony. The model proposed by Deneubourg and co-workers for explaining the foraging behavior of ants is the main source of inspiration for the development of ant colony optimization. In ACO a number of artificial ants build solutions to an optimization problem and exchange information on their quality through a communication scheme that is reminiscent of the one adopted by real ants. ACO algorithms is introduced and all ACO algorithms share the same idea and the ACO is formalized into a meta-heuristics for combinatorial problems. It is foreseeable that future research on ACO will focus more strongly on rich optimization problems that include stochasticity.

/pdf/ant-colony-optimization-artificial-ants-as-a-computational-4d2lxwbp0x.pdf

Ant colony optimization: artificial ants as a computational intelligence technique

https://decsai.ugr.es/~lci/journal-papers-pdf/ijar02.pdf

Ant colony optimization for learning Bayesian networks

Learning Bayesian networks is known to be an NP-hard problem and that is the reason why the application of a heuristic search has proven advantageous in many domains. This learning approach is computationally efficient and, even though it does not guarantee an optimal result, many previous studies have shown that it obtains very good solutions. Hill climbing algorithms are particularly popular because of their good trade-off between computational demands and the quality of the models learned. In spite of this efficiency, when it comes to dealing with high-dimensional datasets, these algorithms can be improved upon, and this is the goal of this paper. Thus, we present an approach to improve hill climbing algorithms based on dynamically restricting the candidate solutions to be evaluated during the search process. This proposal, dynamic restriction, is new because other studies available in the literature about restricted search in the literature are based on two stages rather than only one as it is presented here. In addition to the aforementioned advantages of hill climbing algorithms, we show that under certain conditions the model they return is a minimal I-map of the joint probability distribution underlying the training data, which is a nice theoretical property with practical implications. In this paper we provided theoretical results that guarantee that, under these same conditions, the proposed algorithms also output a minimal I-map. Furthermore, we experimentally test the proposed algorithms over a set of different domains, some of them quite large (up to 800 variables), in order to study their behavior in practice.

Learning Bayesian networks by hill climbing: efficient methods based on progressive restriction of the neighborhood

This paper deals with the problem of supervised wrapper-based feature subset selection in datasets with a very large number of attributes. Recently the literature has contained numerous references to the use of hybrid selection algorithms: based on a filter ranking, they perform an incremental wrapper selection over that ranking. Though working fine, these methods still have their problems: (1) depending on the complexity of the wrapper search method, the number of wrapper evaluations can still be too large; and (2) they rely on a univariate ranking that does not take into account interaction between the variables already included in the selected subset and the remaining ones. Here we propose a new approach whose main goal is to drastically reduce the number of wrapper evaluations while maintaining good performance (e.g. accuracy and size of the obtained subset). To do this we propose an algorithm that iteratively alternates between filter ranking construction and wrapper feature subset selection (FSS). Thus, the FSS only uses the first block of ranked attributes and the ranking method uses the current selected subset in order to build a new ranking where this knowledge is considered. The algorithm terminates when no new attribute is selected in the last call to the FSS algorithm. The main advantage of this approach is that only a few blocks of variables are analyzed, and so the number of wrapper evaluations decreases drastically. The proposed method is tested over eleven high-dimensional datasets (2400-46,000 variables) using different classifiers. The results show an impressive reduction in the number of wrapper evaluations without degrading the quality of the obtained subset.

Fast wrapper feature subset selection in high-dimensional datasets by means of filter re-ranking

This paper deals with the problem of wrapper feature subset selection (FSS) in classification-oriented datasets with a (very) large number of attributes. In high-dimensional datasets with thousands of variables, wrapper FSS becomes a laborious computational process because of the amount of CPU time it requires. In this paper we study how under certain circumstances the wrapper FSS process can be speeded up by embedding the classifier into the wrapper algorithm, instead of dealing with it as a black-box. Our proposal is based on the combination of the NB classifier (which is known to be largely beneficial for FSS) with incremental wrapper FSS algorithms. The merit of this approach is analyzed both theoretically and experimentally, and the results show an impressive speed-up for the embedded FSS process.

José A. Gámez

Papers

Ant colony optimization for learning Bayesian networks

Learning Bayesian networks by hill climbing: efficient methods based on progressive restriction of the neighborhood

Fast wrapper feature subset selection in high-dimensional datasets by means of filter re-ranking

Speeding up incremental wrapper feature subset selection with Naive Bayes classifier

A GRASP algorithm for fast hybrid (filter-wrapper) feature subset selection in high-dimensional datasets