Machine Learning is the study of methods for programming computers to learn. Computers are applied to a wide range of tasks, and for most of these it is relatively easy for programmers to design and implement the necessary software. However, there are many tasks for which this is difficult or impossible. These can be divided into four general categories. First, there are problems for which there exist no human experts. For example, in modern automated manufacturing facilities, there is a need to predict machine failures before they occur by analyzing sensor readings. Because the machines are new, there are no human experts who can be interviewed by a programmer to provide the knowledge necessary to build a computer system. A machine learning system can study recorded data and subsequent machine failures and learn prediction rules. Second, there are problems where human experts exist, but where they are unable to explain their expertise. This is the case in many perceptual tasks, such as speech recognition, hand-writing recognition, and natural language understanding. Virtually all humans exhibit expert-level abilities on these tasks, but none of them can describe the detailed steps that they follow as they perform them. Fortunately, humans can provide machines with examples of the inputs and correct outputs for these tasks, so machine learning algorithms can learn to map the inputs to the outputs. Third, there are problems where phenomena are changing rapidly. In finance, for example, people would like to predict the future behavior of the stock market, of consumer purchases, or of exchange rates. These behaviors change frequently, so that even if a programmer could construct a good predictive computer program, it would need to be rewritten frequently. A learning program can relieve the programmer of this burden by constantly modifying and tuning a set of learned prediction rules. Fourth, there are applications that need to be customized for each computer user separately. Consider, for example, a program to filter unwanted electronic mail messages. Different users will need different filters. It is unreasonable to expect each user to program his or her own rules, and it is infeasible to provide every user with a software engineer to keep the rules up-to-date. A machine learning system can learn which mail messages the user rejects and maintain the filtering rules automatically. Machine learning addresses many of the same research questions as the fields of statistics, data mining, and psychology, but with differences of emphasis. Statistics focuses on understanding the phenomena that have generated the data, often with the goal of testing different hypotheses about those phenomena. Data mining seeks to find patterns in the data that are understandable by people. Psychological studies of human learning aspire to understand the mechanisms underlying the various learning behaviors exhibited by people (concept learning, skill acquisition, strategy change, etc.).

Machine learning

Starts with a brief review of the history and the application areas considered in the literature. The author then proceeds with introductory modeling examples, behavioral and structural properties, three methods of analysis, subclasses of Petri nets and their analysis. In particular, one section is devoted to marked graphs, the concurrent system model most amenable to analysis. Introductory discussions on stochastic nets with their application to performance modeling, and on high-level nets with their application to logic programming, are provided. Also included are recent results on reachability criteria. Suggestions are provided for further reading on many subject areas of Petri nets. >

/pdf/petri-nets-properties-analysis-and-applications-43jwyjdxr0.pdf

Petri nets: Properties, analysis and applications

We propose a simple approach to combining first-order logic and probabilistic graphical models in a single representation. A Markov logic network (MLN) is a first-order knowledge base with a weight attached to each formula (or clause). Together with a set of constants representing objects in the domain, it specifies a ground Markov network containing one feature for each possible grounding of a first-order formula in the KB, with the corresponding weight. Inference in MLNs is performed by MCMC over the minimal subset of the ground network required for answering the query. Weights are efficiently learned from relational databases by iteratively optimizing a pseudo-likelihood measure. Optionally, additional clauses are learned using inductive logic programming techniques. Experiments with a real-world database and knowledge base in a university domain illustrate the promise of this approach.

/pdf/markov-logic-networks-5900hejfpp.pdf

Markov logic networks

Inductive Logic Programming (ILP) is a new discipline which investigates the inductive construction of first-order clausal theories from examples and background knowledge. We survey the most important theories and methods of this new field. First, various problem specifications of ILP are formalized in semantic settings for ILP, yielding a “model-theory” for ILP. Second, a generic ILP algorithm is presented. Third, the inference rules and corresponding operators used in ILP are presented, resulting in a “proof-theory” for ILP. Fourth, since inductive inference does not produce statements which are assured to follow from what is given, inductive inferences require an alternative form of justification. This can take the form of either probabilistic support or logical constraints on the hypothesis language. Information compression techniques used within ILP are presented within a unifying Bayesian approach to confirmation and corroboration of hypotheses. Also, different ways to constrain the hypothesis language or specify the declarative bias are presented. Fifth, some advanced topics in ILP are addressed. These include aspects of computational learning theory as applied to ILP, and the issue of predicate invention. Finally, we survey some applications and implementations of ILP. ILP applications fall under two different categories: first, scientific discovery and knowledge acquisition, and second, programming assistants.

Inductive Logic Programming : Theory and Methods

To successfully apply evolutionary algorithms to the solution of increasingly complex problems, we must develop effective techniques for evolving solutions in the form of interacting coadapted subcomponents. One of the major difficulties is finding computational extensions to our current evolutionary paradigms that will enable such subcomponents to "emerge" rather than being hand designed. In this paper, we describe an architecture for evolving such subcomponents as a collection of cooperating species. Given a simple string-matching task, we show that evolutionary pressure to increase the overall fitness of the ecosystem can provide the needed stimulus for the emergence of an appropriate number of interdependent subcomponents that cover multiple niches, evolve to an appropriate level of generality, and adapt as the number and roles of their fellow subcomponents change over time. We then explore these issues within the context of a more complicated domain through a case study involving the evolution of artificial neural networks.

/pdf/cooperative-coevolution-an-architecture-for-evolving-3p6n3lmfcq.pdf

Cooperative Coevolution: An Architecture for Evolving Coadapted Subcomponents

This paper describes REGAL, a distributed genetic algorithm-based system, designed for learning first-order logic concept descriptions from examples. The system is a hybrid of the Pittsburgh and the Michigan approaches, as the population constitutes a redundant set of partial concept descriptions, each evolved separately. In order to increase effectiveness, REGAL is specifically tailored to the concept learning task; hence, REGAL is task-dependent, but, on the other hand, domain-independent. The system proved particularly robust with respect to parameter setting across a variety of different application domains.

REGAL is based on a selection operator, called Universal Suffrage operator, provably allowing the population to asymptotically converge, on the average, to an equilibrium state in which several species coexist. The system is presented in both a serial and a parallel version, and a new distributed computational model is proposed and discussed.

The system has been tested on a simple artificial domain for the sake of illustration, and on several complex real-world and artificial domains in order to show its power and to analyze its behavior under various conditions. The results obtained so far suggest that genetic search may be a valuable alternative to logic-based approaches to learning concepts, when no (or little) a priori knowledge is available and a very large hypothesis space has to be explored.

Search-intensive concept induction

In this paper we present a concept acquisition methodology that uses data (concept examples and counterexamples), domain knowledge and tentative concept descriptions in an integrated way. Domain knowledge can be incomplete and/or incorrect with respect to the given data; moreover, the tentative concept descriptions can be expressed in a form which is not operational. The methodology is aimed at producing discriminant and operational concept descriptions, by integrating inductive and deductive learning. In fact, the domain theory is used in a deductive process, that tries to operationalize the tentative concept descriptions, but the obtained results are tested on the whole learning set rather than on a single example. Moreover, deduction is interleaved with the application of data-driven inductive steps. In this way, a search in a constrained space of possible descriptions can help overcome some limitations of the domain theory (e.g. inconsistency). The method has been tested in the framework of the inductive learning system, “ML-SMART,”, previously developed by the authors, and a simple example is also given.

A Knowledge Intensive Approach to Concept Induction.

Modeling production rules by means of predicate transition networks

A system that performs automated concept acquisition from examples and has been specially designed to work in noisy environments is presented. The learning methodology is aimed at the target problem of finding discriminant descriptions of a given set of concepts and uses both examples and counterexamples. The learned knowledge is expressed in the form of production rules, organized into separate clusters, linked together in a graph structure. Knowledge extraction is guided by a top-down control strategy, through a process of specialization. The system also utilizes a technique of problem reduction to contain the computational complexity. Several criteria are proposed for evaluating the acquired knowledge. The methodology has been tested on a problem in the field of speech recognition and the experimental results obtained are reported and discussed. >

Automated concept acquisition in noisy environments

REGAL is a Distributed Genetic Algorithm designed for learning concept descriptions from examples, in First Order Logic In particular, each individual in the population represents a conjunctive formula in VL2 language In order to increase the efficiency of the generalization process, REGAL has been provided with a new selection operator, called Universal Suffrage operator, which guarantees (in probability) to maintain a population covering all the learning events As generalization mostly takes place when two individuals covering different sets of examples are crossed, the global generalization capability of the system is increased Moreover, in the case of disjunctive or multiple concepts, the universal suffrage algorithm allows the formation of different species, each one corresponding to a different disjunct In this way, all the disjuncts can be learned in parallel obtaining, in average, more general solutions than by learning them one at a time A formal analysis of the universal suffrage operator is presented, providing theoretical explanations of the experimentally observed behaviour A comparison with the classical selection algorithm and with the sharing function method is also made Finally, a long term control strategy, called “Tories and Whigs”, is proposed in order to overcome the problem of lethal matings between uncompatible disjuncts The effectiveness of REGAL is demonstrated on several learning problems

Attilio Giordana

Papers

Search-intensive concept induction

A Knowledge Intensive Approach to Concept Induction.

Modeling production rules by means of predicate transition networks

Automated concept acquisition in noisy environments

Learning disjunctive concepts by means of genetic algorithms