In the field of machine learning, semi-supervised learning (SSL) occupies the middle ground, between supervised learning (in which all training examples are labeled) and unsupervised learning (in which no label data are given). Interest in SSL has increased in recent years, particularly because of application domains in which unlabeled data are plentiful, such as images, text, and bioinformatics. This first comprehensive overview of SSL presents state-of-the-art algorithms, a taxonomy of the field, selected applications, benchmark experiments, and perspectives on ongoing and future research. Semi-Supervised Learning first presents the key assumptions and ideas underlying the field: smoothness, cluster or low-density separation, manifold structure, and transduction. The core of the book is the presentation of SSL methods, organized according to algorithmic strategies. After an examination of generative models, the book describes algorithms that implement the low-density separation assumption, graph-based methods, and algorithms that perform two-step learning. The book then discusses SSL applications and offers guidelines for SSL practitioners by analyzing the results of extensive benchmark experiments. Finally, the book looks at interesting directions for SSL research. The book closes with a discussion of the relationship between semi-supervised learning and transduction. Adaptive Computation and Machine Learning series

/pdf/semi-supervised-learning-scg1exixez.pdf

Semi-Supervised Learning

Enhancing Text Classification to Improve Information Filtering

Semi-supervised Learning

Humans have an uncanny ability to learn new concepts with very few examples. Cognitive theories have suggested that this is done by utilizing prior experience of related tasks. We propose to emulate this process in machines, by transforming new problems into old ones. These transformations are called metaphors. Obviously, the learner is not given a metaphor, but must acquire one through a learning process. We show that learning metaphors yield better results than existing transfer learning methods. Moreover, we argue that metaphors give a qualitative assessment of task relatedness.

/pdf/teaching-machines-to-learn-by-metaphors-4n1dud4usm.pdf

Teaching machines to learn by metaphors

The gap between abstract and concrete results in machine learning is largely due to unrealistic assumptions made by researchers in formal learning theory. The task of abstractly describing learning is very difficult, so it is natural to make simplifying assumptions. As research progresses, assumptions will slowly become more realistic. Current work in formal learning theory has not yet advanced to the stage where it will be useful to empirical and psychological research in machine learning. Future research in formal learning theory must attack the problem of bias. Bias is essentially a preference ordering on hypotheses. Two forms of bias are suggested here. Recent research in machine learning has examined algorithms that shift their bias for different domains. The biases discussed here resist this trend, as they attempt to apply to all ‘natural’ domains.

The gap between abstract and concrete results in machine learning

The distribution-independent model of (supervised) concept learning due to Valiant (1984) is extended to that of semi-supervised learning (ss-learning), in which a collection of disjoint concepts is to be simultaneously learned with only partial information concerning concept membership available to the learning algorithm. It is shown that many learnable concept classes are also ss-learnable. A new technique of learning, using an intermediate oracle, is introduced. Sufficient conditions for a collection of concept classes to be ss-learnable are given.

/pdf/semi-supervised-learning-4x3g2dmnhk.pdf

Raymond Board

Papers

Semi-Supervised Learning